If anyone fancies actually applying the system to a game, then seeing how the resultant score compares to a score out of 100 you'd give it off the bat, I'd be really interested to see how far out the two scores are.
romeo G. Detlev Jr. wrote:...To give you proper answers: I admit the concept I've suggested is inherently subjective; it just breaks that subjective-ness down into categories rather than asking a reviewer for a subjective whole, for example out of ten. My idea is to come up with something a bit more concrete when comparing games. N64 / NGC Mag was always my go-to as a kid / teenager and they would always rank games based on the score out of 100 they would award. But it was those 1 or 2 % differences that really irked me, particularly when I disagreed with the order they'd result in...
I agree, but the reason percentage scores annoy me is that they are indefensibly precise. I really don't think you can rate a game 92.1% fun as opposed to 90.8% fun, because that implies a difference of 1.3% fun. What is 1.3% fun? That's a very precisely quantified value, but can anyone really describe 1.3 percentage points of fun?
So I'm actually sympathetic to your kind of ensemble of scores, because at least you can trace back a justification for the score itself. But let's think about an even simpler scoring function:
Does the game have good graphics? +1Is the game well-written? +1Does the reviewer find the game to be fun? +1
There are great games out there, like SpaceChem, for which the first two questions aren't even relevant. SpaceChem would get 1/3 in this (it's really fun, but has bad graphics and no story). Meanwhile, a competently written game with decent presentation that is a total slog to play would get 2/3.
I'm not sure there is a set of criteria which are universally applicable as markers of quality across gaming, and I'm not sure that those criteria (if they exist) would be weighted the same across different genres (you'd want 'interesting story' to form a big component of a score for an RPG, but it's more of a small bonus for a racing game).
I still love the old head/heart out of ten scores, back in the day from the Press Start (or was it Push Start? Christ, I forget!) crew from here. Conveyed tons of info with that split.
Pell wrote:I still love the old head/heart out of ten scores, back in the day from the Press Start (or was it Push Start? Christ, I forget!) crew from here. Conveyed tons of info with that split.
Karl wrote:I'm not sure there is a set of criteria which are universally applicable as markers of quality across gaming.
Nor am I - but it's essentially what I'm looking for!
So I'm of the opinion that mere humans can only really, honestly, seriously answer questions about their subjective experiences with "yes," "kinda," and "no" and all of these attempts to use numbers and grades aren't really meaningful. I'm also of the opinion that you can compare general attributes of a game ("is it well-polished?" has applicability to every game) but not specific ones ("are the graphics good?" doesn't have any meaning for a text adventure).
Taking these two thoughts, if you want a score ensemble I could see it working a bit like this: In the reviewer's opinion... No / Kinda / Yes * Have the developers crafted a fun implementation of the core gameplay mechanics? +0 / +3 / +6 * Is the game presented with a high level of polish? +0 / +1 / +2 * Does the game innovate within its genre? +0 / +1 / +2
This gets you your score on a scale of 0-10. At the very least it matches my idea of a 'perfect' game which would be great fun, highly innovative (at time of release, if we're going to judge games retrospectively), and very well polished.
If someone put a gun to my head and forced me to rate games out of 10 that's how I'd do it. But I'd still rather just rate a game on the 'scale' of "It's good!", "Eh, OKish..." or "Pretty bad" to be honest. Because at least that makes it really clear that it's a totally subjective summary of how much you liked it while you were playing it.
Buffalo wrote:I think we’re at a level now that, with a 5 Star system, a 3/5 to me is like ‘no no no’. Which is...bad? Or my call in that I don’t play average games? Interesting.
This might sound a little crazy, but I don't think a star system and a numerical system parse the same. It could be due to the association with hotel classification, but in a star rating system for games (and film), it feels to me more like stars are "awarded" for merit and excellence, rather than simply a mark out of 5. Yes, nothing can get more than 5 stars, but that doesn't mean it's a score out of five.
Something like: ★★★★★ World Class ★★★★ Excellent ★★★ Very Good ★★ Ok ★ Not good
The reason a linear numeric scale doesn't work for critique is because people don't have as many shades of dislike in their repertoire as they do shades of like. You could easily tell the difference between a 7, 8 or a 9, but you'll have a much harder time doing the same with a 2, 3 or a 4. Nobody wants to waste their time with something they don't like, so they tend not to think as much about stuff below what they deem as average.
People who are playing stuff they like in all sorts of varying degrees will then be pushing that stuff to the top end of the scale to distance it from the stuff they can only either a) dislike or b) actively hate
And, over time, anything at least a little fun gets scores 7 and above.
rinks wrote:I would seriously dispute that a longer game automatically deserves a higher score.
Agreed. Games like Civilization and FIFA have always been immeasurable by that metric anyway.
I'd also argue that, for PC games at least, scalability of graphics is an important consideration for some people. A game might look stunning when you've got eight GPUs, but I'd have to take your word for it.
Perhaps abstracts would be an interesting way of providing a brief summary of a reviewer's opinion, whilst not relying on pin the tail on the donkey scoring and making the reader actually read some of the journalist's work.
romeo G. Detlev Jr. wrote:Ok, here is my proposed system. Based on some of the replies in the thread so far, I already know some of you will hate this, but here goes anyway.
Scores will be awarded in seven categories, each with a different maximum possible award. Those categories, and the ratings within them, are:
Visuals Where a rating out of five is given, on the basis that 0 = dreadful 1 = bad 2 = average 3 = good 4 = great 5 = special
Audio Also out of five. Same exact scale as visuals
Level of fun A rating out of twelve, whereby 0 = no enjoyment to be had 1 = some small moments of fun, as a result of the game's poor quality 2 = a decent amount of fun to be had, as a result of the game's poor quality (i.e. so bad it's good) 3 = very few moments of enjoyment 4 = a below average amount of fun 5 = average amount of fun 6 = an above average amount of enjoyment to be had 7 = the majority of the game is fun to play 8 = the game is fun to play save for some minor moments 9 = the game is entirely fun to play 10 = the entire game is very fun to play 11 = every fan of the genre will enjoy this game 12 = every gamer will enjoy this game
Technical quality A rating out of 8, where 0 = broken 1 = barely playable 2 = very poor quality 3 = poor quality 4 = average quality 5 = above average quality 6 = good quality 7 = great quality 8 = mastery of its platform
Game length Out of 4, where 0 = cannot progress at all 1 = a few hours 2 = less than 10 hours 3 = between 10 and 30 hours 4 = over 30 hours
Replay value Out of 4, where 0 = minimal or no replay value 1 = a small amount of replay value 2 = an average amount of replay value 3 = lots of replay value 4 = potentially infinite replay value
Bonus At the reviewer's discretion: 0 = not worth a bonus. 1 = the game is considered very good 2 = the game is considered special
Add the seven scores to get a score out of 40. This score can act as a standalone rating, or be multiplied by 2.5 for a score out of 100, displayed as a %.
So... thoughts?
At the risk of sounding rude, I've got to say I really don't like your proposed system. I think it's far too complicated, and ultimately pointless, (no pun intended) given that you are attempting to apply some sort of scientific measurement to what is, ultimately, a subjective, holistic judgement.
I think that the best reviews make a reasonable attempt to objectively identify a game's essential merits, but also reflect the writer's own predilections.
It might not always be appropriate to comment on how long a game will last or if its graphics are 'special' or merely 'great'- and how is that to be judged, exactly? For all the people who think Wind Waker is unspeakably gorgeous, there are at least as many who think it looks childish and unappealing. So is the judgement based on an objective measurement of FPS, raw polygon count and textures/post-processing, or on a subjective assessment of how the visual design of the game holds together?
A well-written review should read like the score a game is ultimately given. If one were to read the reviews on Metro's GameCentral, for example, they are remarkably consistent in scoring their games in a way that reflects the comments and criticism in the text of the review. EDGE magazine is, in my opinion, slightly less consistent in this regard than it was in the past, but both of those publications use the full 1-10 scale. Thus I wasn't put off buying, for example, 'Space Channel 5', which EDGE gave 6/10 (above average), because the text of the review highlighted its abundant charm while pointing out that it was incredibly slight and ultimately amounted to little more than rote, rhythmic button pressing. It's one of my favourite games. Crazy Taxi got 7/10. Breath of the Wild got 10/10.
The thing with both EDGE and GameCentral is, both would work without scores, because the reviews are well-written (and that is not to say that these are the only examples of good writing, by the way). Looking back at magazines like Mean Machines, or N64, or even Super Play, I get the sense that the scoring was more arbitrary. I can see why it would be irritating that one percentage point would mean the difference being recommended or not; in effect good or crap.
I think Mean Machines was bang on the money a lot of times, but they rarely used the full scale of their scoring system- ultimately anything scoring below 80% tended to be seen as poor, with real stinkers getting sub-50% scores. But they also reviewed themselves into a corner by giving 'Super Mario Bros. 3' 98%. This score then became the high watermark, and was applied to 'Super Mario World' and 'Street Fighter 2', and it was clear they were never going to give a game 100%, because, somehow, that extra 2% would have suggested perfection, in a way that 10/10 simply doesn't.
Super Play, particularly in the early days, seemed speculative rather than definitive in its judgements, and it was not always clear why one game scored higher than another.
N64's secondary ratings system (like Mean Machine's graphics/sound etc scores) was an exercise in futility. Giving a game a score for 'mastery'- how well the reviewer felt the developer had used the hardware, was completely pointless, given that it presupposed that each reviewer was somehow au fait with the development tools and the technical capabilities of the N64. Not once during the N64 era was it pointed out that the chief reason why no decent fighting games and very few racing games of note appeared on N64 was that the N64 couldn't generate anything like the number of raw polygons that the PlayStation could, so I'm not convinced that James Ashton or Jes 'Adverb' Bickham, as much as I enjoyed their writing, had a clue about the N64's challenging dev environment. And if they did, they then chose not to share that with their readership, making the 'Mastery' score doubly pointless.
Ultimately, a review's main aim is to tell the reader whether or not a game is worth spending money on. Nothing more or less. The best reviews do just that.
Attempting to place a game into an ultimate hierarchy is doomed to failure because the hierarchy will continually change as more games are released. Is a game poor, okay, good, great or amazing? Ultimately that's probably what's useful to know. And with the best will in the world, most people don't discuss entertainment in a particularly sophisticated or in-depth way anyway, so an insanely detailed scoring system, as well as being a ball-ache for any reviewer having to use it, is probably going to end up largely ignored too.
That turned out a bit longer than I planned, sorry!
Last edited by That on Tue Oct 10, 2017 1:56 pm, edited 2 times in total.
Reason:fixed broken BBcode
rinks wrote:I think Eurogamer have got it right. Essential, Recommended, (no rating), Avoid.
Spot on. The only other one I like is no scoring at all, like Mafro suggests. However lots of sites won't do either of these because you won't get linked on metacritic.
Scores are useful for quickly determining if a reviewer is positive or negative about a game. I use metacritic a lot, and I find it most useful not for the average numerical score but for collating a load of reviews in one place and displaying them as short paragraph or two line summaries - that's normally all I really want and need to make up my mind about a game. Reading 10 short summaries of reviews is better than reading a single, long review.
Switching scores for a eurogamer system is fine too, as long as you can still get that quick general judgement.
Good podcast (actually more than one) from Tired Old Hack, a.k.a. Chris Scullion, formerly of ONM, CVG and the frankly peerless ONM podcast, which I was an avid fan of, despite never buying the magazine.
OrangeRakoon wrote:Scores are useful for quickly determining if a reviewer is positive or negative about a game. I use metacritic a lot, and I find it most useful not for the average numerical score but for collating a load of reviews in one place and displaying them as short paragraph or two line summaries - that's normally all I really want and need to make up my mind about a game. Reading 10 short summaries of reviews is better than reading a single, long review.
Switching scores for a eurogamer system is fine too, as long as you can still get that quick general judgement.
I go with Metacritic too, and here sometimes. Metacritic mauled Mafia 3 largely but I love it. Lots of reviews mentioned bugs though which I believe have been patched out.
I had no idea about that stat regarding the N64 and fighting games. And you're more than welcome to tear into my system!
