Discussion about a reform on LDCs

[~MP3 Amplifier~]

Don't worry I wasn't believing for a second that you were saying I was wrong, after all, this is all opinion. :p I think Yuri's idea of using this judging technique for a duel is the best idea.

[Harmless]

Ah.

I have my doubts that it would flow smoothly, but if they're willing to try might as well let them.

Mind if I also judge the duel?

[~Yuri]

Ah.

I have my doubts that it would flow smoothly, but if they're willing to try might as well let them.

Mind if I also judge the duel?

If Supershroom is really willing to do that duel, then I see no problem with any of you guys judging, as his system will require a bigger amount of judges than the usual.

[Doram]

This post by Doram was thumbed up by: 2

CedarBranch (September 16th, 2014, 8:36 am) • MessengerOfDreams (September 16th, 2014, 8:21 am)

[Harmless]

insert beautiful doram logic here

/thread

[Supershroom]

Meanwhile we've also been talking about something else, right? Something about what makes the difference between hard and frustrating levels, and that hard levels are difficult and controverse to judge? So MoD, BY, Harmless, Nwolf, MK or whoever else to dislikes the reform right now, this is a perfect example why I advocate for it.

I hate how one person rates a level "9/20" and another rates it "17/20."

I hate it too, and I guess everyone else hates it too but he just doesn't want to say it. Like I've said already, if it ever comes to such a situation, there's something wrong. People are getting upset about it, and it causes drama, hatred whatsoever. That just needs to stop. We can't prevent it from happening, but we can prevent it from being valid.

This is not the Olympics. This is not Politics. This is not World Peace or World War.

Sure it isn't, but professional LDing appears to me to be more than just a game. It's both passion, art, science and mental exercise.

The goal of choosing a winner is picking a level that is different or awesome or weird and showing that it is different than the average.

That's exactly what I've tried to do, doing many unconventional and weird things I've never seen before, stuff like the boat ride on hot water, the flame dodge between two narrow walls, jumping from one metal cap to the next etc. and all that. And I've got heavily punished for it, basically because it was too much, but I was also surprised that some nice little details weren't rewarded with bonus points in "Other", e.g. only MoD recognizing the SMW reference. But the point of what I want to say is that a perfect mixture of conventionality and unconventionality makes a great level in many cases.

Pure averages are beautiful for a reason. Guess what. First judge is having a good day? He scores everybody high? Ok. The final scores run from 15 to 20. Second judge is a bad day? He scores everybody low? Ok. The final scores run from 10 to 15. The third judge is having a normal day. She judges everybody evenly. The final scores run from 12 to 18. Average it out. Guess what? The person who got a 20 from one, a 15 from the second, and an 18 from the third won. That's because they were the highest in their brackets. That's what the averaging does. That is the point of averaging.

If it's meant to be a contra, it isn't. Like I've said, judges must not be influenced by their current mood, or at least limit that as much as they can. And despite from this situation being very theoretic, as long as all judges have the same level rankings, the winner stays the same no matter in which intervalls each judge's scores are. But the problem is that rankings can be heavily varying, JJJ's level was the top level for MoD and the third worst one for Harmless, and the intervals can also be very patchy, with sometimes distances of more than 4 points gapping between two adjacent levels. 6 judges, 4 results for each level being valid, so we keep our beautiful averaging, but with less stupid distortions.

[~MP3 Amplifier~]

This post by -BY was thumbed up by:

~MP3 Amplifier~ (September 22nd, 2014, 9:40 am)

[MessengerOfDreams]

This post by MessengerOfDreams was thumbed up by:

CedarBranch (September 22nd, 2014, 10:20 am)

[Nwolf]

everybody is forced to play SMW and know references okay

[Doram]

Ok, fine, my list wasn't definitive. Let's continue, then.

Elaboration on point 3) This is a game.
I will point out that, after what you quoted, my very next words were that for many people this IS serious business. Yes, this is something that is a passionate task for many people, and as such is something that a lot of thought, feeling, and action is centered around. My point is that we should not let that passion cloud our judgement of whether the process is "fair" or not, purely because we may have done poorly, or well for that matter. Objective decisions need to be made objectively, dispassionately, and calculatedly, to be the most "fair" for everyone. And just to nip the next rebuttal in the bud, I say dispassionately, but I do not mean dismissing passion as a factor. I merely recommend that passion be put in its place as a fact of the process, and consider the passionate aspects of other people's contributions without letting your own passion get in the way of your own judgements.

Elaboration on point 4) Exceptional is the point.
Yes, I understand that you are trying to be unique. I am not denying that fact at all. What I am trying to point out, through this and other points, is that a level's uniqueness is further refined by how well it communicates its uniqueness to an audience. It is all well and good to come up with something that you consider cool and unique, but it is something entirely different to get everyone who plays your level to see it as cool and unique as well. They have to understand that uniqueness, and then they have to agree that it is unique. This boils down to the fact that the act of playing a level is art at its highest function, in that it is a conversation between the artist and the viewer, and video games are unique in the human arts because they are interactive, and that conversation is usually an extended and complex one. If your communications skills are lacking, then the most unique idea in the world will fail. If people cannot figure out what is going on, either by the mechanism not being refined enough to be consistent, or the example not being constructed well enough to express its intent and properly showcase the mechanism, then the uniqueness attempt fails. This is part of judging any level on uniqueness. Not only is it a matter of "Is this unique?", but also "Is this clearly and properly set up?", "Does it consistently work?", and "Is it clear to every player what is going on?"

Elaboration on point 6) Pure averages are beautiful for a reason.
Yes, the pure averages work. Part of the reason they work is because they replicate and represent the personal biases of any random player that attempts to play your level in the future. Some people like platformers. Some people do not. If you take an average of every private vote on a submitted level and average THAT out, you should get the same kind of rating that you will get from a panel of judges. This is because some people will like your level because of its adherence to certain principles , like being a good platformer, and other people will not like it for it's non-adherence to other principles, like having a good story. Those two opinions are not related at all, except for the fact that they will both apply to the same level in an aggregate called "public opinion". Average all THAT out, and you will get exactly what we are already getting. A majority agreement on some aspects, and proper deviance for specific critiques in a wide range of related subject matters. The elaboration for point 4 touches on this in that not every judge will be able to properly critique certain aspects of the level, just as not every normal member of the internet will be able to properly critique it., Again, the goal is to take all these widely differing ideas, ideals, and skillsets into account together, and come up with an aggregate "score".

Elaboration on point 8) Every system has it's flaws.
Yes, this is an incredibly complex and complicated process that we undertake here, from design to judging. Yes, the scores are an incredibly condensed form of opinion on any given level. Yes, that distillation and compression dismisses or loses some complexity in the compression process. No, the process cannot be made any more decompressed without becoming unworkable. The judges are already asked to play EVERY level in a contest, ALL the way through, regardless of skill level, and then they are asked to express, as best they can, EVERY quality of the level that they see, as plainly and clearly as possible. This takes time. This takes effort. I dare say that there is equal amount of work put into judging a contest as there is put into submitting a level for the contest. When that effort becomes MORE than it takes to submit a level (and that is generally considered at least a month's worth of free time), then you are asking too much of the process, and more importantly, its participants. Frankly, the demand for judging times to be reduced as much as possible has been slightly unfair for exactly this reason.

Let's back up for a moment, and discuss the wonders of the judging process. Most people have heard the "statistic" that we only use 10% of our brain power for everyday tasks. This is mostly true, but ignores a very important aspect of how the brain works: the subconscious. The subconscious runs through the rest of the 90% with unfettered access to much more data and capability than the conscious brain ever needs to worry about. And that is the crux of this. Just because that 90% isn't being consciously controlled, doesn't mean that it is sitting there completely useless. It is chewing on everything that goes on around you just as your consciousness does, and it comes to it's own conclusions and makes its own decisions, which are integrated into the totality during dreaming when your unconscious state allows the conscious and subconscious to meet each day. Now, just because that meeting only happens once a day, doesn't mean that the subconscious doesn't have it's ability to speak up when it feels the need. The primary tool of the unconscious is emotion, and the secondary tool is instinct. Emotion is a distilled understanding of the totality of the situation, and instinct is the subconscious decision about what to do about it. You will feel a certain way about something and "not know why" because you have come to certain conclusions about the situation using the other 90% of your brain that the 10% completely missed. "Gut feeling", "hunch", "intuition", whatever you want to call it, you have more ability to deal with the world than you will ever know, and the smartest people will recognize that, and the most capable people will make use of it.

Now let's zoom back in on the task at hand: judging levels. The good judge can make up a "Score" based on their total understanding of the level, both conscious and subconscious - taking into account the totality of their experience as gamers playing this game and others, their ability to communicate and understand external communication (both in terms of navigating the communication of playing your level, and being able to express the resulting decision), and their ability to correlate all that with what they understand of the public at large, and its desires, pleasures, frustrations, and communication levels. A bad judge can fail at any aspect of this, but only insomuch as the "average gamer" will, and that brings us back into point 6.

Am I saying this system is perfect? No. Am I saying that it is probably about as perfect as we can get it? Yes. I have literally witnessed YEARS of arguing, headdesking, and learned discussion over this, and I have contributed as much as I can to the process, as has everyone else who is passionate about level design. We have eliminated ambiguities, streamlined the process for speed and accuracy, and attempted to ensure fairness in every way at every point. All that is left is to plug the available humans at the time into the process and cross our fingers. The process is only as fallible as humanity in general, and there is only one thing to be said to that: Good luck to us all.