The problem with reviewing MMORPGS

This article is on one of my pet hates - mainstream gaming websites like Gamespot and IGN reviewing MMORPGs. Some of you will instantly know why I find their reviews infuriating, others might need a few words of explanation. First out, I consider the idea of reviewing an MMORPG in a few pages as frankly ridiculous, and secondly I think the big websites serve to slow progress of the genre.

The first claim is easy enough to explain: MMORPGs are huge games. Now, I'm aware that MMORPGs tend to get reviews that run to five or six pages, much like high profile games like GTA4 or Metal Gear Solid or Half Life 2 all get, and I'm also aware that there are massive offline games too (the aforementioned GTA4 or Baldur's Gate 2, for example), but I refuse to believe and of the reviewers releasing a review a week after the release of an MMO have actually played up to max level and extensively tested raiding and PvP. Instead, they're just giving you their impressions of the game up to mid-level, during a period where the novelty value is still strong. In short, nothing that actually resembles what playing the game is actually like. Another part of the problem lays in the fluidity of all the content in an MMO. If I boot up Baldur's Gate 2 and have a play, it'll be exactly the same as it was five years ago. If I boot up World of Warcraft or Everquest and go to the newbie zones, the experience will be completely different now to how it used to be, so all the WoW reviews written even six months ago are probably outdated now.

This isn't just due to the endless tinkering with class skills and new zones/zone revamps, but it does play a big part. One of my favourite zones in the original Everquest was the Lake of Ill Omen, because it was huge, open, well populated and itemised, varied and simply beautiful. Lots of players felt the same, and in the Velious era that I primarily played in (often regarded as the golden age of EQ) it was always bustling. When the Shadows of Luclin expansion came out, it became faster to level up in the new zones, so the population dropped off sharply. Suddenly I couldn't group in my favourite zone so much, which made vast areas of it inaccessible and changed my playing experience a great deal.

It was usually MUCH busier than this, believe me.

Similarly, before Luclin came out there was a huge tunnel in the Western Commonlands where everyone used to gather and sell their wares (I'm not sure why all the world's traders congregated in that particular spot, but they did). Everyone would shout about their wares and prices in general chat, and business would be conducted by sending a tell to the person in question and then finding them and exchanging money for the item. Luclin brought with it the Bazaar, the first incarnation of the system that would later grow into WoW's Auction House, and predictably this instantly killed the informal market in the WC tunnel. It changed the game a great deal - trading became more convenient, but I did rather like the human interaction of the Commonlands tunnel. That's an aside, though - my point is that the market in the tunnel was a social phenomenon, and even if you went back to EQ now you wouldn't be able to experience it. An MMO is a game that's constantly in flux, a game where if you miss something then it is gone forever. A review will never be more than a snapshot, which is why it infuriates me that major sites treat them like normal games.

To compound matters, there is social churn. At its most basic, the same people won't be online all the time, so one person might really enjoy a dungeon run because they had a competent and entertaining group, while another person might find the same dungeon tests his patience because he tried it with a group of idiots. To an extent, guilds mollify this point, but the fact that a great deal of an MMO is the social experience is something that most reviewers seem to ignore. You can't really review a playerbase besides huge and somewhat useless generalisations (WoW is full of ten-year-old griefers, EQ2 has a friendly and grown up playerbase). There is some truth to them, but that doesn't guarantee that everyone will have the same experience - and, worse, the social side of an MMO changes even faster than the game world.

In a lot of ways, the idea of giving a game a score is ridiculous anyway. Sure, if a game has massive design flaws then it might be worse than a game that doesn't, but if you have two well-known games it just comes down to preference. Is Half Life 2 better than Crysis? I'd argue not, but plenty of people would disagree. It just depends what you like. Slapping a score of 9 on WoW and an 8 on EQ2 implies that WoW is objectively the better game, but that's far too simple a picture. If you like depth, you'd do much better with EQ2. If you're looking for a consistently designed game with good accessibility, play WoW. The best a review can ever do is give a feel for the game, to let you see if you think you'd like to play it or not. That's why I put together my play diary for EQ2 and started the one for AoC - a normal review just doesn't give a good enough indication of how a game actually works. So I wish everyone would stop doing them.

A much better use of the Source physics engine than the stupid gravity gun.

The second allegation in my post was that the mainstream gaming sites hold back innovation in the genre. My simple argument for this is the fact that they only cover the biggest names in the business, which is partly understandable given how many MMOs seem to be in development, but this naturally means that the games that get the most coverage will always big the huge-budget efforts by Blizzard or Funcom or SOE or BioWare. In my mind, these guys will always play a bit safe with their games because they're spending a fortune on them and (with the possible exception of SOE) they all have a good reputation they don't want to soil by releasing a game that gets critically panned. And therefore they're not too keen on taking risks, on breaking new ground - something every genre needs. Some of the best games I've played are distinctly quirky, like Portal for example. The game Portal was based on was made by a small group of students who were hired by Valve (who are pretty left-field anyway) in the same way they hired the creators of Counterstrike. I just don't see one of the big studios taking a risk and releasing something a little different, a little quirky.

And, in essence, that's why seeing Gamespot and IGN trying to cover the latest MMOs makes me angry.