Discussion about a reform on LDCs
Moderator: Level Moderator
Re: Discussion about a reform on LDCs
by Harmless » September 12th, 2014, 1:03 pm
How do we determine who is experienced though?
~ Tesla Bromonovich
-
Harmless - Is it lunch time yet?
- Posts: 2793
- Joined: June 25th, 2011, 11:53 am
- Location: Mother Russia!
Runouw Votes Winner - Thumbs Up given: 271 times
- Thumbs Up received: 240 times
Re: Discussion about a reform on LDCs
by MessengerOfDreams » September 12th, 2014, 1:22 pm
Has designed for a few LDCs is the best way. They'll know the ropes, what a good level looks like, what's expected of them as judges.
-
MessengerOfDreams - Moderator
Error contacting Twitter - Posts: 6615
- Joined: August 16th, 2009, 11:31 am
- Location: When I figure it out, I'll let you know.
Winter - Thumbs Up given: 519 times
- Thumbs Up received: 707 times
Re: Discussion about a reform on LDCs
Thumbs up x1 
by nin10mode » September 12th, 2014, 1:47 pm
The only problem I ever had with judging was inexperience.
Naturally, I'm referring to my butthurt over some 16th and 18th LDC judgings, where I received 7/10s in fun with no explanation as to what kept it from a perfect 10, and to a lesser extent, the 10th LDC because there were no suggestions for how to bring the seasonal change mechanic further, only that it was lacking.
So I'm all for a few regular panels. I actually wouldn't mind adding all these other methods to this main one. An experienced panel that excludes the lowest and highest really doesn't mean much except maybe canceling out some very unlikely bias, and anonymous levels has the same effect, but to a greater extent. Of course, these panels would have to be phased out every once in awhile.
Naturally, I'm referring to my butthurt over some 16th and 18th LDC judgings, where I received 7/10s in fun with no explanation as to what kept it from a perfect 10, and to a lesser extent, the 10th LDC because there were no suggestions for how to bring the seasonal change mechanic further, only that it was lacking.
So I'm all for a few regular panels. I actually wouldn't mind adding all these other methods to this main one. An experienced panel that excludes the lowest and highest really doesn't mean much except maybe canceling out some very unlikely bias, and anonymous levels has the same effect, but to a greater extent. Of course, these panels would have to be phased out every once in awhile.
-
nin10mode - Immune to Death
Error contacting Twitter 
Error contacting last.fm - Posts: 3087
- Joined: October 3rd, 2009, 6:11 am
- Location: Not here
Credit To Team - Thumbs Up given: 102 times
- Thumbs Up received: 221 times
Re: Discussion about a reform on LDCs
by MessengerOfDreams » September 12th, 2014, 1:56 pm
How did you not bring up 6/20?
-
MessengerOfDreams - Moderator
Error contacting Twitter - Posts: 6615
- Joined: August 16th, 2009, 11:31 am
- Location: When I figure it out, I'll let you know.
Winter - Thumbs Up given: 519 times
- Thumbs Up received: 707 times
Re: Discussion about a reform on LDCs
by NanTheDark » September 12th, 2014, 3:57 pm
Do I count as being experienced?
I have a ton of Wikipedia and TV Tropes articles about videogames read, a lot of Extra Credits, Sequelitis and videogame reviews seen, including the Great Clement's 3 hour long review of Sonic '06, as well as having over a decade of experience playing various kinds of videogames.
...Not to mention I've designed a couple of levels for LDCs.
inb4 no
I have a ton of Wikipedia and TV Tropes articles about videogames read, a lot of Extra Credits, Sequelitis and videogame reviews seen, including the Great Clement's 3 hour long review of Sonic '06, as well as having over a decade of experience playing various kinds of videogames.
...Not to mention I've designed a couple of levels for LDCs.
inb4 no
-
NanTheDark - As it breaks, it becomes stronger
- Posts: 2018
- Joined: December 23rd, 2010, 6:04 pm
- Location: Your screen
Runouw Votes Winner - Thumbs Up given: 191 times
- Thumbs Up received: 217 times
Re: Discussion about a reform on LDCs
by Supershroom » September 12th, 2014, 11:39 pm
Nwolf wrote:if a level is not fun for a judge and he also can't see technical skill in it, it's a valid reason to give it a low score.
Excuse me but if no other judge also gives a low score for it, then why do the other 4 or 5 judges all enjoy the level and give higher scores? Why are all other judges not able to see the flaws the first judge has meant to see? Same with a very high score - if all other judges give a bit lower scores and point out flaws to justify them, it rather means the first judge hasn't minded them.
MessengerOfDreams wrote:I was a super happy judge so I gave high scores.
Excuse me again but does that mean one could lose some places just because a judge has been in a bad mood on the day he's judged his level? I try to judge as independent from my current mood as I can. If it's meant to be like this, then it's rather a pro than a contra.
Aznchau4ever wrote:Maybe I'm not just reading it right, but what is deciding which score gets chosen to be removed?
After the 5 or 6 judges have given their scores, the highest and the lowest scores get removed for each entrant.
Harmless wrote:How do we determine who is experienced though?
As I've applied for judge, Nwolf gave me some levels from previous LDCs to judge as a test.
Last edited by Supershroom on September 13th, 2014, 5:47 am, edited 1 time in total.
-
Supershroom - Our local SM63 Spielberg
- Posts: 1802
- Joined: June 21st, 2014, 5:22 am
- Location: Filmstudio for SM63. Sometimes also LL.
Runouw Votes Winner - Thumbs Up given: 244 times
- Thumbs Up received: 126 times
Re: Discussion about a reform on LDCs
by *Emelia K. Fletcher » September 13th, 2014, 1:51 am
Yes, you are definitely absolutely aware of the fact that pEOPLE ARE ALL DIFFERENT AND HAVE DIFFERENT CRITERIA FOR A GOOD LEVEL
-
*Emelia K. Fletcher - Who's this douchebag?
Error contacting Twitter Error contacting last.fm - Posts: 2926
- Joined: July 24th, 2010, 3:40 am
- Location: A\//\\/A
Cookie - Thumbs Up given: 42 times
- Thumbs Up received: 211 times
Re: Discussion about a reform on LDCs
Thumbs up x3
by ~MP3 Amplifier~ » September 13th, 2014, 4:03 am
I'm late to this topic, very late.
(I was too ill to look at a bright screen yesterday, sorry)
Judging is a subjective job, I think people just have to come to terms with that. Humans in general are subjective, especially when you're judging on fun, isn't it about how fun you find the level to play?
E.g. Space Colony is one of the few levels I would probably give a 10/10 of Fun in, because I enjoyed it that much, and even then I would try to be objective- if I saw any gameplay that incorrectly used or wasn't really the same standard as everything else, I'd probably go down to a 9.75, which is still very high but not perfect. I'm an instinctive person, I trust my gut. I don't think I was a judge for Nin's 18th LDC level, but if I had judged it and given it an 8.75-9/10 in Fun (which is probably what I would have given it), but I didn't really give any full criticism about the gameplay, Nin would wonder "hang on, what is holding my level down enough to keep it 1.25 points away from the perfect score?". It would be because of my gut feeling, no not the gut feeling to be biased against him, but I've been playing levels long enough now that I get certain "vibes" that I can't describe. It's just, I play a level, and I can immediately guess what Fun score I'd give it, what Graphics score I'd give it etc, without even going into details. When I start to weigh out the good and the bad objectively, 98% of the time the score remains the same. This is probably the upside of me judging for so long.
I know I'm rambling on about my view on judging, but I've been mentioned a lot here as being one of the generous judges in the more recent LDCs, when I used to be one of the stricter ones. This is not because I feel sorry for all of you and want you to get higher scores than me. This is because I've been judging more objectively recently. I might dislike a level, but if the gameplay ideas are technically good and accurate or have good ideas that could be executed extremely well with the right advice, then I will still give it a score of 6.5 or above in Fun, unless of course I really truly hated the level. However back in the 16th LDC when I judged the first time, I was just like "I don't like your level lol so have a really low score from me <3". But even back then I was still as objective as I could possibly be at the time.
To me, my individual score of Shroom's series (14/20) was perfectly justified. The level dragged, lagged, got annoying and too repetitive; but if he'd executed his ideas a bit better, because he did have good ideas, he would have had a much higher score from me. On the objective side of things, I still gave him a 7 something out of 10 in Fun because I wasn't too fond of it, but it was technically not bad at all. In some ways, I marked him down more because he used too many ideas. If you get lots of really good gameplay ideas, you should use them for different levels, not all in the same series, because otherwise how do you move on from that?
tl;dr Judging is subjective, what one judge will think is a good level, one might think is bad. We're all human, we can't all be 100% objective. If you can't handle the truth, then that's your problem not ours.
I don't mean to come across harsh, I'm just being honest from my point of view.
(I was too ill to look at a bright screen yesterday, sorry)
Judging is a subjective job, I think people just have to come to terms with that. Humans in general are subjective, especially when you're judging on fun, isn't it about how fun you find the level to play?
E.g. Space Colony is one of the few levels I would probably give a 10/10 of Fun in, because I enjoyed it that much, and even then I would try to be objective- if I saw any gameplay that incorrectly used or wasn't really the same standard as everything else, I'd probably go down to a 9.75, which is still very high but not perfect. I'm an instinctive person, I trust my gut. I don't think I was a judge for Nin's 18th LDC level, but if I had judged it and given it an 8.75-9/10 in Fun (which is probably what I would have given it), but I didn't really give any full criticism about the gameplay, Nin would wonder "hang on, what is holding my level down enough to keep it 1.25 points away from the perfect score?". It would be because of my gut feeling, no not the gut feeling to be biased against him, but I've been playing levels long enough now that I get certain "vibes" that I can't describe. It's just, I play a level, and I can immediately guess what Fun score I'd give it, what Graphics score I'd give it etc, without even going into details. When I start to weigh out the good and the bad objectively, 98% of the time the score remains the same. This is probably the upside of me judging for so long.
I know I'm rambling on about my view on judging, but I've been mentioned a lot here as being one of the generous judges in the more recent LDCs, when I used to be one of the stricter ones. This is not because I feel sorry for all of you and want you to get higher scores than me. This is because I've been judging more objectively recently. I might dislike a level, but if the gameplay ideas are technically good and accurate or have good ideas that could be executed extremely well with the right advice, then I will still give it a score of 6.5 or above in Fun, unless of course I really truly hated the level. However back in the 16th LDC when I judged the first time, I was just like "I don't like your level lol so have a really low score from me <3". But even back then I was still as objective as I could possibly be at the time.
To me, my individual score of Shroom's series (14/20) was perfectly justified. The level dragged, lagged, got annoying and too repetitive; but if he'd executed his ideas a bit better, because he did have good ideas, he would have had a much higher score from me. On the objective side of things, I still gave him a 7 something out of 10 in Fun because I wasn't too fond of it, but it was technically not bad at all. In some ways, I marked him down more because he used too many ideas. If you get lots of really good gameplay ideas, you should use them for different levels, not all in the same series, because otherwise how do you move on from that?
tl;dr Judging is subjective, what one judge will think is a good level, one might think is bad. We're all human, we can't all be 100% objective. If you can't handle the truth, then that's your problem not ours.
I don't mean to come across harsh, I'm just being honest from my point of view.
The below image is a montage of my individual highest placing LDC creations, as a reminder to myself that level designing is a part of my life that I can't just leave behind
-
~MP3 Amplifier~ - THE DARK LAMP
Error contacting Twitter - Posts: 4383
- Joined: May 6th, 2011, 12:35 pm
- Location: Maaars d(^_^)b
SM63 Level Designer Contest Winner - Thumbs Up given: 226 times
- Thumbs Up received: 611 times
Re: Discussion about a reform on LDCs
by MessengerOfDreams » September 13th, 2014, 4:11 am
I was a super happy judge because there were levels that were do good that by left me happy and I'm very complimentary of levels that make me feel that way.
-
MessengerOfDreams - Moderator
Error contacting Twitter - Posts: 6615
- Joined: August 16th, 2009, 11:31 am
- Location: When I figure it out, I'll let you know.
Winter - Thumbs Up given: 519 times
- Thumbs Up received: 707 times
Re: Discussion about a reform on LDCs
by Supershroom » September 13th, 2014, 5:45 am
Well, maybe I've made the impression to say that my idea would be an overall perfect solution and it would bring ultimative fairness and zero unfairness. I've never meant to say something like that. Judging will forever be subjective. But heavy deflections are odd as well. No matter if the judges don't participate or the levels are anonymous, they will still happen here and there. They need to be curtailed. There are pro's and contra's for my idea, but still I'm sure the advantages overbalance the disadvantages. It's not a perfect solution, but a kind of approach or harmonisation.
I still have the wish to test it out. Instead of a mini-LDC, maybe we could also have another double-LDC like the 21st, and test it on one half, while for the overall results of both LDC's, we still use the common system.
I still have the wish to test it out. Instead of a mini-LDC, maybe we could also have another double-LDC like the 21st, and test it on one half, while for the overall results of both LDC's, we still use the common system.
-
Supershroom - Our local SM63 Spielberg
- Posts: 1802
- Joined: June 21st, 2014, 5:22 am
- Location: Filmstudio for SM63. Sometimes also LL.
Runouw Votes Winner - Thumbs Up given: 244 times
- Thumbs Up received: 126 times