Request for Transparency

Level Designer Contests, discussion about and help with the Level Designer, information about glitches, and levels and level series can be found here.

Moderator: Level Moderator

Final judge discussion?

Yes
10
63%
No
6
38%
 
Total votes : 16

Re: Request for Transparency

Postby Harmless » May 5th, 2015, 7:36 pm

I completely agree ven, but my problem is not 'judges don't communicate enough and peer review others for any mishaps', but rather, 'what if the judges completely disagree on a topic already readily checked by both respective judges?'. For all you know, one judge could've thought a platforming segment was the greatest thing they ever played, and another judge could completely disagree and say it was frustrating, repetitive, and was a bad difficulty spike.
Expect something cool here soon!

~ Tesla Bromonovich
User avatar
Harmless
Is it lunch time yet?

 
Posts: 2793
Joined: June 25th, 2011, 11:53 am
Location: Mother Russia!

Runouw Votes Winner
For winning Master of a Hidden Talent in the RV Summer 2017

Thumbs Up given: 271 times
Thumbs Up received: 240 times

Re: Request for Transparency

Postby MessengerOfDreams » May 5th, 2015, 7:37 pm

Yeah, my problem is the other three trying to mold the one to submission despite having valid points.
Image
Image

My Most Recent Works: show
I switch my signature a lot. If you wanna see some of my past ones, here you go.
Silent Conversations and a Crow's Final Song!
My latest story, and one of my personal favorites. A girl bound in silence finds the words to say to her prospective girlfriend as they visit her religious father in a dusty town on the edge of Kansas, where the crows' migration south brings forth anchored memories, the path to resolution, and a new start.

Form (25quared)
This might be like nothing you've seen before.
Updated Works! Some of my past best and current stuff: show
Writing Works!
Fanfic: Shut Up and Dance
why do I write so much about dancing you don't dance you've never danced in your entire life
lying little ♥♥♥♥ with your ♥♥♥♥ story ♥♥♥♥ you
also Diddy/Lucina <3

Fanfic: Worth a Thousand Words
Because the world needed a Samus/Dedede story
Fanfic: Ecstatic Silence
Just wanted to write and ♥♥♥♥ like this happens, you'd think I'd know better.
Fanfic: Far From the Edge
It's a dance that's been a long time coming for a brand new man and an unchanging woman, but once one takes the plunge there's no falling back up.
Original: Jealous Ghosts of the Mississippi
The story of Rachel meeting Amber after a lifetime of silence and being shunned
Original: Your Hand in Mine
One of my most personal stories about a dangerous romance. Now to be published in a college lit journal!
-------------------------
Recent LDC Work:
Level Series: Leaves From The Vine (ft Star King)
For the 29th LDC, a theme of grassland taken through the growth of our designing society. Won the 29th LDC!
--
Levels at Large:
Level Topic: Collection of MoD's Levels!
If you've ever wanted to see any level I made worth a damn, go here! From the quiet 14th LDC entrant Finis to the megasmash level series Dark, you can find links here!
"You were always a revolutionary, now there's just less of a chance of you crying in the corner." ~Ridder
User avatar
MessengerOfDreams
Moderator

Error contacting Twitter
 
Posts: 6615
Joined: August 16th, 2009, 11:31 am
Location: When I figure it out, I'll let you know.

Winter
2016 Story Contest

Thumbs Up given: 519 times
Thumbs Up received: 707 times

Re: Request for Transparency

Thumbs up x1

Postby Harmless » May 5th, 2015, 7:40 pm

I suppose it wouldn't hurt to try though. Now that I think about it, like Ven said, if the one can defend his point (providing examples ofc), then maybe the other three are needing to recheck their scores?

Though it's still a very tricky situation no matter what. I think for now we should just stick to 'judges discuss and share their thoughts, don't quite try to change other scores yet but bring up valid criticism if the others have missed it'.
Expect something cool here soon!

~ Tesla Bromonovich
User avatar
Harmless
Is it lunch time yet?

 
Posts: 2793
Joined: June 25th, 2011, 11:53 am
Location: Mother Russia!

Runouw Votes Winner
For winning Master of a Hidden Talent in the RV Summer 2017

Thumbs Up given: 271 times
Thumbs Up received: 240 times

Re: Request for Transparency

Postby SuperMIC » May 6th, 2015, 11:19 am

Another problem, although mostly unlikely, is the issue of bias. Given the fact that judges are allowed to enter LDCs, there could be some inevitable bias with judges trying to nudge their levels a little bit. Unfortunately, I can speak from experience during WITBLOAT judgings.

That being said, the WITBLOAT judging process is efficient and effective and everyone's usually happy at the end. So maybe we could give this a try.
<Arocado:> I'm Asian.
PROOF.

Image
^ Panik, you rock. ^

RUNOUW: DANGER AND DECEPTION: Read it! You could be in it!

"Surveys": show
Image
Image

Image
User avatar
SuperMIC
The Legacy

 
Posts: 692
Joined: February 2nd, 2011, 12:08 pm
Location: Br... it's chilly here, eh?

Thumbs Up given: 85 times
Thumbs Up received: 60 times

Re: Request for Transparency

Postby Supershroom » May 6th, 2015, 11:45 am

SuperMIC wrote:Another problem, although mostly unlikely, is the issue of bias. Given the fact that judges are allowed to enter LDCs, there could be some inevitable bias with judges trying to nudge their levels a little bit.

I don't think any of our reliable judges would go like "oh, this level will be a strong contender for me, let's look for -0.25's I can dock off".

Even the often-quoted Chau in the 6th LDC didn't do that for his own gain, he just has/had very strict standards and he also had them in other LDC's he didn't enter.
Image

Thank you, FF.

Nan's sig! From the previous Secret Santa event!: show
Rating standards: show
1 star: (0 - 5.75 in LDC) These levels need a lot of work. They're totally un-elaborated, and/or inherently flawed due to things like cutoff or enemy spam, or they're untested and there are too many bugs gameplay-wise, in short: Many many things went wrong in such a level.

2 stars: (6 - 9.75) These levels are not terrible, but poor. They're too short, lack scenery or they have errors and/or bugs but it's still barely enjoyable. Again, spend more effort and try to do better. (if it's your first level that I rate two stars, it's not bad. There's a long way you can still go)

3 stars: (10 - 12.75) These levels are about mediocre. They may be still a little bit short, the gameplay is fine but not very original and graphics are also solid, but not breath-taking. Try to keep improving!

4 stars: (13 - 15) These levels are nice, but not the best. Effort has been put into them, there can some professionality be seen in gameplay and graphics, they're a nice experience to play, but there are still things you can do to make it even better. Try to go the extra mile!

5 stars: (15.25 - 20) These are really great levels, those which, as said, I would give more than 15 in an LDC. These suffice many high requirements, and it already goes into subtleties if you want to make them better. These levels are successful all around. Bravo.

These are just general principles I try to follow when rating. Sometimes I don't leave a further comment when rating, most times I do, if you have questions about it, feel free to ask further.
Look at this!: show
My total SM63 level experience: 2803 levels.
Total coins collected in these: 306800

The WITBLO11 Tribute


A showcase of the 25 best levels of 2011, including such fantastic levels as Niveau Victorieux Galaxy, Plastic Beach, Dimension Infinity, Realm of the Heavens/Parallel Spires, and more!

My Level Collection: http://runouw.com/forums/viewtopic.php?f=25&t=47186

Other Important Links:
Last Legacy Level Designer tutorial
Eternal SM63 Medal Table
Eternal SM63 Score Table (all LDC scores available)
Top 10 Lists for LDC's (highest averages in a single judging criterion)
Runouw Votes Eternal Table
User avatar
Supershroom
Our local SM63 Spielberg

 
Posts: 1802
Joined: June 21st, 2014, 5:22 am
Location: Filmstudio for SM63. Sometimes also LL.

Runouw Votes Winner
New Years 2016: Best Level Winner (Ride Through The Mines)

Thumbs Up given: 244 times
Thumbs Up received: 126 times

Re: Request for Transparency

Postby ~MP3 Amplifier~ » May 6th, 2015, 2:46 pm

Supershroom wrote:I don't think any of our reliable judges would go like "oh, this level will be a strong contender for me, let's look for -0.25's I can dock off".


This is one of the main things, really. People's opinions on the judgings they receive heavily depend on if you trust the judges or not. Personally, I think if someone has been given permission to judge, people need to put their faith in them otherwise it's only inevitable that conflict will break out at some point.




I've entered a lot of LDCs and in nearly every LDC I can remember, there was always one score that I really didn't like and wished could be higher.
This is perfectly natural and it's not the judges' fault, they were just doing their job. If it's a really unfair point reduction, then fair enough, if we go through with these meetings, hopefully we can knock out any of those before releasing the results. But if it's down to pure opinion then I think anything has a right to stay the same if that's how the player feels about that level.
"I treat everyone equally, depending on how much I like them." ~Me
The below image is a montage of my individual highest placing LDC creations, as a reminder to myself that level designing is a part of my life that I can't just leave behind

Image
I made this sig so credits to mee :amp smile:
User avatar
~MP3 Amplifier~
THE DARK LAMP

Error contacting Twitter
 
Posts: 4383
Joined: May 6th, 2011, 12:35 pm
Location: Maaars d(^_^)b

SM63 Level Designer Contest Winner
LDC #30: Finale/The Ultimate Battle

Thumbs Up given: 226 times
Thumbs Up received: 611 times

Re: Request for Transparency

Postby Harmless » May 6th, 2015, 4:26 pm

Now that I think about it, backup judges serve the purpose of judging the Judges's levels so we don't have another accidental chau.

So technically the 6th LDC shenanigans shouldn't be a worry.
Expect something cool here soon!

~ Tesla Bromonovich
User avatar
Harmless
Is it lunch time yet?

 
Posts: 2793
Joined: June 25th, 2011, 11:53 am
Location: Mother Russia!

Runouw Votes Winner
For winning Master of a Hidden Talent in the RV Summer 2017

Thumbs Up given: 271 times
Thumbs Up received: 240 times

Re: Request for Transparency

Thumbs up x1

Postby nin10mode » May 7th, 2015, 2:11 am

See, but having backups has its own set of problem. The first problem is that whatever scale they're using only applies to 1-4 people. If they typically judge from 15-20 while the normal judges judge from 10-17, guess what that means?
Image
Art: show
Anime: show
Imagei use mal now but this sigbar is pretty
Videos: show
ImageImage
im@s: show
Image
User avatar
nin10mode
Immune to Death

Error contacting Twitter
Error contacting last.fm
 
Posts: 3087
Joined: October 3rd, 2009, 6:11 am
Location: Not here

Credit To Team

Thumbs Up given: 102 times
Thumbs Up received: 221 times

Re: Request for Transparency

Postby Harmless » May 7th, 2015, 9:21 am

That's a good point. So maybe discussion can occur between judges about a level not including the Judge who made the level? I think that's the best course of option.
Expect something cool here soon!

~ Tesla Bromonovich
User avatar
Harmless
Is it lunch time yet?

 
Posts: 2793
Joined: June 25th, 2011, 11:53 am
Location: Mother Russia!

Runouw Votes Winner
For winning Master of a Hidden Talent in the RV Summer 2017

Thumbs Up given: 271 times
Thumbs Up received: 240 times

Re: Request for Transparency

Postby -BY » May 7th, 2015, 11:17 am

You guys ok with closing the poll by end of this week?
Image

Big thank you to FrozenFire who created this masterpiece. : )
User avatar
-BY
Honorary Member

 
Posts: 900
Joined: May 23rd, 2012, 11:43 pm
Location: 298

Razzian Fighter

Thumbs Up given: 72 times
Thumbs Up received: 100 times

PreviousNext

Return to Super Mario 63 Level Designer