Official SM63 Judging System Discussion

[Oranjui]

Numerous complaints have been raised to the staff concerning bias in the current "Overhaul of SM63 Judging System" thread, so we've collectively decided to create a new official thread for a restructure of the system. This thread is designed to facilitate a focused and moderated discussion, weighing all voices equally and not placing a preference on any specific proposal. The OP of this thread will keep an archive of the discussion topics, as well as links to the beginning of each new discussion topic in the thread.

Keep in mind that as staff and the actual hosts of official LDCs, we'll consider the outcome of the discussions and have the final say ourselves. This means we may take some ideas, and we may tweak others as deemed necessary.

Let's begin, shall we?

---

Archives

The "Other" Category": show

The "Other" Category

Concerns have arisen over the validity of the Other category, over both fairness and bare necessity. That is--is the Other category fair, or does it need to be changed? is the Other category necessary at all, and could we remove it or replace it with something else? Would changing the Other category cause too much of a change in judging standards, requiring a refactoring of all past LDC scores? A list of the most popular options proposed to us has been compiled below. As the discussion seems to be coming to a close, a poll will be put up.

1) Keep Other as it currently stands
2) Merge Other with Graphics and/or Gameplay, and create a general all-encompassing "Atmosphere" category to replace Graphics+Other
3) Keep Other but replace the current additive scoring system with a holistic overall score for anything not included in Gameplay or Graphics
4) Remove the Other category, and replace it with a new Originality category that uses a holistic overall scoring system out of 5
5) Keep Other as it is but give it a much smaller weight, similar to LL's scoring system--likely out of 2 (5→2) points--and raise the weight of Graphics accordingly (5→8)
6) I have a different thought that's not represented here, let's discuss more before we vote
Note: In this context, "Holistic" means to consider the level as a whole and give a rating out of 5(?). "Additive" means to start at a default value like 3/5(?) and use + or - to tweak that default score.

What would you like to see happen with the Other category? (select all that apply/top few options)
Poll ended at Tue, 17 May 2016 15:29:29

Option	Votes	Percent
No changes		24%
Remove the category		12%
Merge with Graphics to create "Atmosphere"		20%
Replace the additive score with a holistic score		32%
Give it a different scoring weight		0%
Replace with a new category like "Originality"		12%
Something else (specify in the thread)		0%
	Total votes: 25

<Next Topic>

Let's talk.

[npromin1]

I'm not that good when it comes to plot and story, but if a well-executed plot-line is in a level, it deserves extra points. Points shouldn't be taken away because there is a plot.

I say 2, or merge "other" with "graphics" to create "atmosphere" so both categories are worth 10 points. Under the current system, most of the judges add points to the "other" category for things like plot or music, which ends up affecting the atmosphere of the level anyway.

Either 10/10, return to the original system (Points in the other category tend to be there for good music and plot anyway), or no judging criteria at all.

[Harmless]

Why merge "Other" with Gameplay and Graphics categories though? "Other" is the literal definition of "Other" and "Miscellaneous": Anything that's separate from the above topics stated. Bugs, any memorable moments in the level, anything that doesn't really have to do much with overall Gameplay/Graphics? (And as you might've guessed, yes, a level contains a lot more detail than gameplay and graphics - They can contain ingenious ideas, creativity, so much more notable stuff that I feel like gets overlooked.)

Personally I think the judging system is fine as it is. I don't really see anything that needs immediate attention. And if there have been concerns arisen about the "Other" category (or just bias for the whole system in general), can you post them OJ so we have more context?

edit: Oh, I just realized this was also more of a continuation off Supershroom's thread a week ago. While I haven't really read the whole thing, I'm personally against an overhaul of the "Other" category.

[npromin1]

Anything that's separate from the above topics stated. Bugs, any memorable moments in the level, anything that doesn't really have to do much with overall Gameplay/Graphics? (And as you might've guessed, yes, a level contains a lot more detail than gameplay and graphics

I'm thinking of changing Graphics to Atmosphere (This includes music, plot, and the general feeling surronding the level) and merging it with other. I'm also sure that bugs or creative things can go under gameplay. And I'm fine with the current judging system, rather then something other then this.

[nin10mode]

Why merge "Other" with Gameplay and Graphics categories though? "Other" is the literal definition of "Other" and "Miscellaneous": Anything that's separate from the above topics stated. Bugs, any memorable moments in the level, anything that doesn't really have to do much with overall Gameplay/Graphics?

The philosophy behind removing Other is that these things can arguably be placed in the main categories. The idea behind this revamp is to repurpose the first two categories so they encompass everything that Other has.

If you expand Graphics into including the story and music, it becomes a category commenting on how aesthetically pleasing an entry is overall; essentially, how artful a level is.

Bugs are objectively flaws in gameplay, as much as platforming is a given in gameplay. Innovations such as intentional glitches could be considered extensions of good platforming or puzzles.

They can contain ingenious ideas, creativity, so much more notable stuff that I feel like gets overlooked.

I have a problem with these general terms, so if you could expand on that for people on my side that plan on continuing discussion, it would most likely help.

[Harmless]

Creativity and making clever resourceful use of the level designer's functions do, in fact, stand alone from things that aren't merely "Gameplay" or "Aesthetically pleasing" or "graphics".

Remember Jumbo's level A Goomba Tale, where he managed to make a full fledged, animated title screen of sorts by tricking out the level designer's capabilities (putting Mario under the stage out of camera sight yet still high enough to not go past the bottomless pit below, allowing a goomba to 'follow' him but make it look like it was moving to the right). This sort of thing was never done before AND it was well executed - it set the stage for the rest of the level really well. I'm not the only one to praise this either, just about every other judge for the 30th LDC gave points in "Other" for that, because not only was it something nobody had ever seen before, but it was just downright amazing that one could make a 'title screen' of sorts that set the stage for the level. If you were to award points for this kind of creativity, would you put it in Gameplay or Graphics? ... You wouldn't really put it in either, right?

And yes, Bugs and glitches could be noted as (un)intentional flaws in the gameplay, but they could also be referring to graphical glitches as well, which some designers have taken advantage of instead of letting it be a hindrance in the level. Item disabling - a noteworthy glitch used by many level designers to this date, is more geared towards the Graphical aspect of a level. I don't think that deserves to be in Gameplay, as obvious as that might sound, yet they are still notes about specific bugs/glitches used in the level regardless. I feel like the term 'Bugs/Glitches' encompass more and aren't restricted to just being gameplay elements.

If we want to be really technical, there could be instances in levels where disabled items/poorly layered graphics interfere with the gameplay (such as blocking visibility of the player/platforms ahead). But that could just as easily fit in "Other" as it could in "Gameplay".

If we're going to be stuffing tons of general criteria into bigger criteria points and making them so general to the point where they're called "Atmosphere", then I might as well just write all of my reviews under one criteria: "Pros and Cons". Both giving reviews and creating levels has so much more going for them than just placing down a few platforms and testing how good the 'gameplay' is, or how beautiful these 'graphics' look. There's much more to talk about in levels than these two subjects - Storyline (does anyone even remember the best part of MoD's famous level series Dark, or MP3 Amplifier's series Tearing Paper?), Replayability (If you ask me, I could go back and replay Buffooner's Shyguy Islands for the 20th LDC right now and still enjoy it), Flavor Text/Comedy (A positive level has worked wonders to keep the player engaged), Creativity (It's something I always try to strive for, though whether or not it leads me to being successful is another story), etc.

But that's obviously to not discredit Gameplay or Graphics at all! At heart, SM63 is a platformer - and the platformer genre is notorious for... well, platforming, fun, execution, and being simplistic in a sense. And that's not bad at all. But when you look at platformer titles you'll notice the big names tend to be unique. Super Mario Bros during its time was considered super unique, simplistic, fun, and very, very replayable. Super Mario 64 took Mario to a 3D world where the developers could do more with the game/series. LittleBigPlanet decided to let us be the creators and open our imagination to endless possibilities. FEZ took 2D puzzles to a whole new level with changing your perspective around what was actually a 3D world. I could go on, but... well, I'm sure you get it by now. Assuming enough heart is put into the level, any level made from anyone can stand out from the crowd, and be judged by something that is more than just 'gameplay' and 'graphics/atmosphere'. And while it may vary from judge to judge, I still believe there is something that we can still keep an eye out for.

Keep the Other category, if only for that.

[Karyete]

If I'm honest, the only problem I have with Other is that it's in shorthand. If the first two categories have detailed explanations of the judgings, why not do the same for Other?

[Supershroom]

Okay guys, let's get some things straight once more (even though the respective posts were nuked)

Yes MoD, you haven't been really constructive and you probably didn't listen when we tried to explain to you that your designing style wouldn't be condemned to die under any system.

And yes Nan, you've provoked her incautiously.

Both is not terrible to have. You just need to sit up yourselves.

And most importantly, this IS intended to be a democratic process. When a majority decides on something, it's not the staff that should reserve the final say for itself, except for maybe deviations in terms of a special LDC theme. Also, I rather would have the complaints being directed towards me in order to handle them, instead of moving to another person's thread without knowing what's exactly going on.

Also, I've already mentioned that while using a holistic system for Other would improve things, it still leaves potential for doublings, severe untransparency or other cryptic stuff to write.

"Gameplay" is a fairly general term indeed, and I would definitely book Jumbo's cutscene innovation under that. Yet the separation between Gameplay and Atmosphere (artfulness) is a rather discrete one. Each simple review without categories would eventually split at that point automatically.

So, 12/8 is what I would keep voting for, and 10/10 I would also accept. But everything else is not good to have IMO.

And Harmless, if you read everything from the previous thread, I'd appreciate it.

[Harmless]

"Gameplay" is a fairly general term indeed, and I would definitely book Jumbo's cutscene innovation under that. Yet the separation between Gameplay and Atmosphere (artfulness) is a rather discrete one. Each simple review without categories would eventually split at that point automatically.

Yes, it's general, but it's not like it encompasses literally everything that came to exist about platforming. Same thing goes with "Atmosphere" - I remember NintendoCapriSun (a youtuber) talking about that word once and saying how it's literally one of the most useless words in existence just because of how over-general it is. And to some extent, I actually do kinda agree with him. It's over general to the point where you might as well just have one criteria point that's worth 20 points in a level.

And I really don't see why a cutscene/animated title screen would fall under Gameplay. Explain in more detail, please.

[Venexis]

And I really don't see why a cutscene/animated title screen would fall under Gameplay. Explain in more detail, please.

Just off the top of my head but, does it contribute toward the player's actions? Does it give the player any motivations, hints, tips? Hints and guidelines are flat out gameplay, but it's also possible to encompass the player's decision to follow Choice A over Choice B under that umbrella as well. It's the player deciding, not some arbitrary plot compulsion or forced linearity, right?