Overhaul of SM63 judging system

[Megar]

I've always been of the idea that judges should just straight up not participate so none of this backup judging/potential bias problems can happen

[Supershroom]

Well, there are some points raised against backup judging ... probably you'll need a lot of self-confidence into your own judging scheme being balanced if you take a backup position. If you don't want to do that, the easiest thing is applying for full judge directly :>

Also OJ, I'll first let the poll run out until I see that nothing will change (which could be around the 26th or such) and then I'll put everything together and set up a poll against the old system. I'm really hoping to find a majority, it would be SUCH a pity if all this effort would go to waste.

And judges shall be allowed to participate simply for the reason that they usually recruit themselves mostly from experienced and committed designers. If you prohibit potential judges to participate, you'll almost never get your panel full. (EDIT: Since I actually aim at entering every LDC in existence, you don't want me to never judge again, do you?)

[-BY]

To keep it short:

I see no point in using two judging systems. I'd either stick to one of them. Especially shouldn't vary within the contest itself.
Everyone should judge everything with an equal system. Otherwise it's literally anarchy. D :

Why using 12/8 if 15/10 looks more fabulous? More points might also offer more difference between levels. Additionally maybe avoiding decimalception.

The whole backup judge thing has always been one of the greatest problems with the LDC score stuff here.
Since there's not enough people, there's probably no real solution for that.

Music affects the mood your level covers. Wether you want it to be true or not. That's why I did, do and will pay attention to that.
And it will always influence the way I'm judging the level to some extend. Imagine playing all your favourite levels while they're mute. Personally. I couldn't imagine getting the same feel from them, since I'd feel that something would be missing. Apart from "Hah, Huh, Yippiiiieeeh", "Ufff" and "Dooowww" of course.

[Supershroom]

Okay, here are the cliff notes on what I'd like to see.

Rough descriptions

Gameplay: Score out of 12

Is the level fun and entertaining to play in general? Does it have much of a replay factor, and does it master the art of platforming, puzzling or scavenger-hunting or alike? Does it have a consistent degree of difficulty, or is it frustrating at times, or is it boring and unoriginal? Are there any bugs or other technical problems? Is there something very special and innovative? Do loading times and eventual lag have an influence on the gameplay?

Atmosphere: Score out of 8

How do the overall graphics of the level feel like? Does it have an artistic appeal, is it a coherent and well-balanced work? Are there any obvious flaws like cutoff or messy item placement? Is there a lack of tile variation décor (plants, fences, rocks) where there should be, or is it overworked so it becomes confusing? Furthermore, how do other atmospheric aspects affect the level? Is there a nice music chosen, or does the music choice ruin it? Is there an interesting plot or an amazing storyline that fits with what the level offers?

Guidelines

• Judges should play each level until they have figured it out sufficiently. Normally at least two playthroughs are required (at least I always do that).
• Judges should be aware of their playing skill. The question "is this segment too difficult / too unforgiving" is the easiest source of controversy, that's what an ldcjudgings channel is really important for. If a level indeed suffers from bugs or too much difficulty, yet shows effort and interesting stuff, you shouldn't penalize it too much.
• Lag and loading time deductions / bonuses should be kept to a reasonable limit. If a level is small, so it requires heavy tampering to give it a large loading time, it doesn't really deserve a bonus for no loading time. If a level has a large or even insane loading time, check the cause. Have items been used really wastefully, or is it just the theme of the level which requires all kinds of décor? Similar stuff goes for lags - is there blatant spam of enemies or 30-coin-blocks, or is there just no way to avoid the transition being tall or crowded? If you can't come up with a proper idea on how to lift or ease the lag/loading time issues, a large deduction (1 point or more) is not really justified.
• Story and music, whether it's in-game or external, should be usually considered for Atmosphere. You can vary its emphasis, but the recommendation for it is 1.5 points each (the people also have spoken on that). Actually, there can be barely a plot at all (e.g. pure platformer), in this case, you can just omit it entirely if you want to.
• Round your scores to quarters. (Without Other, this is also a lot easier to follow)

Further comments

Under these circumstances, judges will stay different individuals, and you can still generally judge the way you like, you don't have to change much. Freedom of judging style stays granted while formal inconsistency can, and should be prevented at the same time. There HAS to be a uniform way to go; we always went by the same formal system during 30 LDCs, but sometimes people have undermined its barely defined guidelines to do really stupid stuff when judging, especially in Other, and THAT used to cause controversy and anarchy.

At least, two different scalings in the same LDC is not beneficial at all. There can be room for switching to 10/10 on a music-or-story-based LDC theme upon the host's / the team's discretion, but 12/8 (or 3:2, expressed as proportion) should be the standard thing. If you're having issues with the numbers being not divisible by 5, you can alternatively do it as 15/10 and it will be hit down from 25 to 20 points.

I wouldn't mind actually if scores are increasing a little. With the new system we probably still have to wait a long time (or even infinitely) for the first perfect 20 from a single judge. But more 18s and 19s would be nice. There is no need at all to re-do previous LDCs and decide if Parallel Spires would have won to Destinations with the new system. Past is past.

I'm also starting to fully realize the issues about backup judges. We've had a few LDCs where they were missing and the judges participating had 3 scores while everyone else had four. It would cause a few bias either way, but omitting a backup judge is probably the better choice (unless you would desperately need one since you only have 3 full judges, like in the 5th mini-LDC). Using #ldcjudgings is a MUST to take care of questionable scores and larger discrepancies (5 points or more); even if it delays the results a little, it's worth the effort.

So now, a new general poll is there. Hoping for enforcement.

[Megar]

Absolutely in support of the new system

[~Yuri]

For a better SM63 judging system, and for the sake of simplicity,

Eu voto sim! (though I prefer the 10/10 system rather than the 12/8)

[MessengerOfDreams]

This post by nin10mode was thumbed up by: 2

MessengerOfDreams (April 28th, 2016, 2:47 pm) • TheExcavator (April 29th, 2016, 3:27 pm)

[npromin1]

Honestly,I like 10/10 better, but 12/8 isn't too bad.

[MessengerOfDreams]

10/10 is far better. 12/8 just gives an advantage and bias to people whose specialty is gameplay, which is suspect. But we don't get that option in the polls...

[Supershroom]

The current poll just asks you if you want a 2-category system in general. And for that it looks pretty good ...

Just as you say that 12/8 would give advantage to the gameplay specialists, others actually say that 10/10 would give even more advantage to specialists on graphics and atmosphere. I'll consider putting another poll on that after this one