Features

Published on July 11th, 2016 | by Dean Love

2

Our ratings system and review guidelines

Having written a good deal of reviews over the past few weeks, I’ve noticed that my room ratings, on a five star scale, skew a fair bit lower than many of our contemporaries.

There’s a reason for that, which I figured I’d explain now rather than wait until the first angry e-mail from a room owner who thinks their three read like a four. Essentially, I’m using the rating system taught to me by the first editor I ever had on a professional reviewing job. It  works like this:

1 Star: a really bad game, with few to no redeeming features.
2 Stars: a decent game that makes a few big mistakes, or a bad game that does few things really well, or has some really interesting concepts.
3 Stars: a good game. This is the most important one. A 3 star review is a recommendation. It either means I really enjoyed it, or I found enough interesting in it despite the flaws to be recommending it.
4 Stars: a near perfect game. This is more than a recommendation, this is saying the game is brilliant and everyone should play it. They’re games I unconditionally recommend.
5 Stars: a near perfect game that then does something more. There are, as of writing, three 5-star reviews on the site. The only reason there are that many is selection bias (if I hear a game is really good, I’m more likely to want to go play it). These three games happen to have either the best set design, best single puzzle or best technology I’ve ever seen in a game (at time of playing).

There are no half stars. If a game really does sit somewhere between the two then we round to the middle, so 1.5 is 2 stars, 2.5 is 3 stars, 3.5 is 3 stars, 4.5 is 4 stars.

So there you go. Another way to look at it is it’s essentially a 4 star scale, with 5s basically being unattainable unless you do something truly incredible.

Next, of course, I am an enthusiast. I love playing escape games. They are generally “a good experience”. And very few are actively bad to the point of being broken. Compared to what you do in the rest of your life, even if you go play a two star room, you’re probably going to have a four star experience. An Escape Review 3 is a Trip Advisor 5.

Also worth noting: I’m reviewing the experience I had playing the game, not what the game could be in a different set of circumstances. Bad customer service and broken props will bring the score down, even though the owner might be having a bad day or the team before smashed something in two. Equally, maybe happenstance creates a truly magical moment in the room that can’t be reproduced, it’s still accounted for in the score. I can only review the experience I have, not the theoretical one I might have (though of course, I’ll mention this in the review text). This also means that reviews are a snapshot in time. Rooms change, for the better (feedback leading to better pacing and difficulty) or worse (puzzles breaking and not being replaced). The one small exception to this, is that while I write the reviews, and they’re my opinion, that opinion will naturally be influenced by the opinion of the rest of the team in post-game discussions.

Lastly, we don’t delete comments unless they’re rude or inappropriate, so if you’re an owner and want to respond to the review, that’s the best place to do it. We won’t ever edit reviews after posting other than to correct factual inaccuracies.


About the Author

Dean is a professional writer who has worked for The Mail On Sunday, The Digital Fix, MicroMart and others.



2 Responses to Our ratings system and review guidelines

  1. I wish I’d thought this through more carefully when I first started my blog. Actually I did but not well enough! I think you’ve got a better approach, although I’d argue that you set the five star boundary a little high (at least based on your description). Compressing the “just don’t play this” into a single star category makes sense. I’m wasting some of my scale by splitting that category into “truly terrible” and “beginners might think this is OK, but don’t go here”. While they do mean something different, they’re not really useful to the reader.

    I *do* update the stars on my review occasionally. Rewriting history might be wrong, but my main focus is to guide people to the right rooms. I’ve not made any major changes, but I’ve moved the odd room up or down as I’ve compared it against the currently available games.

    • Dean says:

      I don’t even really like stars at all but know how popular they are! The five-star boundary is very high on purpose. On the site I used to work for they’d review 200+ shows in a year and give two or three the 5 stars. A couple of years there weren’t any. It’s a bit unfair as sometimes something will blow me away in a given game then I’ll see it done by someone else later on and it’s less impressive. But it’s designed to capture those things that truly innovate.

      But yeah, I have the advantage of having worked with ratings scales and lot and had many, many discussions about them. And other than the binary play/don’t play system, the 4*+1 seems to be my favourite.

      I’d considered if ratings should be revised – although I’ve said not the exception would be if I actually replayed a game and it had changed. But given most rooms will change somewhat over time, the “moment in time” approach works best for me.

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to Top ↑