Request for Transparency

Level Designer Contests, discussion about and help with the Level Designer, information about glitches, and levels and level series can be found here.

Moderator: Level Moderator

Final judge discussion?

Yes
10
63%
No
6
38%
 
Total votes : 16

Re: Request for Transparency

Postby Harmless » May 7th, 2015, 9:57 pm

As long as there's still discussion, might as well keep the poll open.
Expect something cool here soon!

~ Tesla Bromonovich
User avatar
Harmless
Is it lunch time yet?

 
Posts: 2793
Joined: June 25th, 2011, 11:53 am
Location: Mother Russia!

Runouw Votes Winner
For winning Master of a Hidden Talent in the RV Summer 2017

Thumbs Up given: 271 times
Thumbs Up received: 240 times

Re: Request for Transparency

Postby Supershroom » May 12th, 2015, 7:02 am

So this is done deal, am I right?

However, I'm still a bit concerned about going like

Venexis wrote:15, 17.5, 16, 17, 16, 16.5. Yup, looks good, next level.

Even if the scores are as narrow as this, it still leaves space for unfair deductions within the range of up to one point or more, which could be decisive in an LDC as narrow as e.g. the 25th with 6th place up to 3rd place being in the range of a quarter point. That's why I've said judges should also utter their main reasons for deductions.

On the other side, it's at least a good benefit to even out those massive score discrepances.
Image

Thank you, FF.

Nan's sig! From the previous Secret Santa event!: show
Rating standards: show
1 star: (0 - 5.75 in LDC) These levels need a lot of work. They're totally un-elaborated, and/or inherently flawed due to things like cutoff or enemy spam, or they're untested and there are too many bugs gameplay-wise, in short: Many many things went wrong in such a level.

2 stars: (6 - 9.75) These levels are not terrible, but poor. They're too short, lack scenery or they have errors and/or bugs but it's still barely enjoyable. Again, spend more effort and try to do better. (if it's your first level that I rate two stars, it's not bad. There's a long way you can still go)

3 stars: (10 - 12.75) These levels are about mediocre. They may be still a little bit short, the gameplay is fine but not very original and graphics are also solid, but not breath-taking. Try to keep improving!

4 stars: (13 - 15) These levels are nice, but not the best. Effort has been put into them, there can some professionality be seen in gameplay and graphics, they're a nice experience to play, but there are still things you can do to make it even better. Try to go the extra mile!

5 stars: (15.25 - 20) These are really great levels, those which, as said, I would give more than 15 in an LDC. These suffice many high requirements, and it already goes into subtleties if you want to make them better. These levels are successful all around. Bravo.

These are just general principles I try to follow when rating. Sometimes I don't leave a further comment when rating, most times I do, if you have questions about it, feel free to ask further.
Look at this!: show
My total SM63 level experience: 2803 levels.
Total coins collected in these: 306800

The WITBLO11 Tribute


A showcase of the 25 best levels of 2011, including such fantastic levels as Niveau Victorieux Galaxy, Plastic Beach, Dimension Infinity, Realm of the Heavens/Parallel Spires, and more!

My Level Collection: http://runouw.com/forums/viewtopic.php?f=25&t=47186

Other Important Links:
Last Legacy Level Designer tutorial
Eternal SM63 Medal Table
Eternal SM63 Score Table (all LDC scores available)
Top 10 Lists for LDC's (highest averages in a single judging criterion)
Runouw Votes Eternal Table
User avatar
Supershroom
Our local SM63 Spielberg

 
Posts: 1802
Joined: June 21st, 2014, 5:22 am
Location: Filmstudio for SM63. Sometimes also LL.

Runouw Votes Winner
New Years 2016: Best Level Winner (Ride Through The Mines)

Thumbs Up given: 244 times
Thumbs Up received: 126 times

Re: Request for Transparency

Thumbs up x1

Postby MessengerOfDreams » May 12th, 2015, 7:51 am

...what do you expect? Someone literally to be punished for having any different scores?
Image
Image

My Most Recent Works: show
I switch my signature a lot. If you wanna see some of my past ones, here you go.
Silent Conversations and a Crow's Final Song!
My latest story, and one of my personal favorites. A girl bound in silence finds the words to say to her prospective girlfriend as they visit her religious father in a dusty town on the edge of Kansas, where the crows' migration south brings forth anchored memories, the path to resolution, and a new start.

Form (25quared)
This might be like nothing you've seen before.
Updated Works! Some of my past best and current stuff: show
Writing Works!
Fanfic: Shut Up and Dance
why do I write so much about dancing you don't dance you've never danced in your entire life
lying little ♥♥♥♥ with your ♥♥♥♥ story ♥♥♥♥ you
also Diddy/Lucina <3

Fanfic: Worth a Thousand Words
Because the world needed a Samus/Dedede story
Fanfic: Ecstatic Silence
Just wanted to write and ♥♥♥♥ like this happens, you'd think I'd know better.
Fanfic: Far From the Edge
It's a dance that's been a long time coming for a brand new man and an unchanging woman, but once one takes the plunge there's no falling back up.
Original: Jealous Ghosts of the Mississippi
The story of Rachel meeting Amber after a lifetime of silence and being shunned
Original: Your Hand in Mine
One of my most personal stories about a dangerous romance. Now to be published in a college lit journal!
-------------------------
Recent LDC Work:
Level Series: Leaves From The Vine (ft Star King)
For the 29th LDC, a theme of grassland taken through the growth of our designing society. Won the 29th LDC!
--
Levels at Large:
Level Topic: Collection of MoD's Levels!
If you've ever wanted to see any level I made worth a damn, go here! From the quiet 14th LDC entrant Finis to the megasmash level series Dark, you can find links here!
"You were always a revolutionary, now there's just less of a chance of you crying in the corner." ~Ridder
User avatar
MessengerOfDreams
Moderator

Error contacting Twitter
 
Posts: 6615
Joined: August 16th, 2009, 11:31 am
Location: When I figure it out, I'll let you know.

Winter
2016 Story Contest

Thumbs Up given: 519 times
Thumbs Up received: 707 times

Re: Request for Transparency

Thumbs up x1

Postby ~MP3 Amplifier~ » May 12th, 2015, 8:08 am

I think just a quick discussion and a look into each judging before the results are released in each LDC, will be enough. If there are no unfair point reductions, then there are no reasons for scores to change. Yes, being objective is important as I've said so many times in the past, but if you were 100% objective and had no opinion of your own to share then that is even worse.

Like, for me a very fun level with only a couple of gameplay flaws would equal a high 8, low 9 score. For some, that might equal almost a perfect 10, just down to a 9.75. If everyone ignored their opinions, then it would be totally pointless. People have just got to find a good balance between the two, and okay if someone found a level very fun but had only a few flaws, and then gave that level a 6/10 in Fun, that should be picked up on, just like if they were marked down for a bug that they can't control like layering issues etc.

So yeah, I personally see no issues with the judging itself, I just think we should review them to stop people carelessly adding or taking off points without realising what the actual reason is.
"I treat everyone equally, depending on how much I like them." ~Me
The below image is a montage of my individual highest placing LDC creations, as a reminder to myself that level designing is a part of my life that I can't just leave behind

Image
I made this sig so credits to mee :amp smile:
User avatar
~MP3 Amplifier~
THE DARK LAMP

Error contacting Twitter
 
Posts: 4383
Joined: May 6th, 2011, 12:35 pm
Location: Maaars d(^_^)b

SM63 Level Designer Contest Winner
LDC #30: Finale/The Ultimate Battle

Thumbs Up given: 226 times
Thumbs Up received: 611 times

Re: Request for Transparency

Postby Supershroom » May 12th, 2015, 8:27 am

~MP3 Amplifier~ wrote:People have just got to find a good balance between the two, and okay if someone found a level very fun but had only a few flaws, and then gave that level a 6/10 in Fun, that should be picked up on, just like if they were marked down for a bug that they can't control like layering issues etc.

So yeah, I personally see no issues with the judging itself, I just think we should review them to stop people carelessly adding or taking off points without realising what the actual reason is.

You've described perfectly what I've meant. Many deducts fall under the subjectivity that a judge has a right on, but negligence is avoidable as well as being too nit-picky when giving the 6/10 as mentioned.

Also I can't understand why you would feel punished if a mistake of yours would be clarified, since you want to be fair to all contestants, right? Then again, it's a maturity task of not being too smart-alecky when picking up someone else's judgings, and we should be constructively willing on finding compromises.
Image

Thank you, FF.

Nan's sig! From the previous Secret Santa event!: show
Rating standards: show
1 star: (0 - 5.75 in LDC) These levels need a lot of work. They're totally un-elaborated, and/or inherently flawed due to things like cutoff or enemy spam, or they're untested and there are too many bugs gameplay-wise, in short: Many many things went wrong in such a level.

2 stars: (6 - 9.75) These levels are not terrible, but poor. They're too short, lack scenery or they have errors and/or bugs but it's still barely enjoyable. Again, spend more effort and try to do better. (if it's your first level that I rate two stars, it's not bad. There's a long way you can still go)

3 stars: (10 - 12.75) These levels are about mediocre. They may be still a little bit short, the gameplay is fine but not very original and graphics are also solid, but not breath-taking. Try to keep improving!

4 stars: (13 - 15) These levels are nice, but not the best. Effort has been put into them, there can some professionality be seen in gameplay and graphics, they're a nice experience to play, but there are still things you can do to make it even better. Try to go the extra mile!

5 stars: (15.25 - 20) These are really great levels, those which, as said, I would give more than 15 in an LDC. These suffice many high requirements, and it already goes into subtleties if you want to make them better. These levels are successful all around. Bravo.

These are just general principles I try to follow when rating. Sometimes I don't leave a further comment when rating, most times I do, if you have questions about it, feel free to ask further.
Look at this!: show
My total SM63 level experience: 2803 levels.
Total coins collected in these: 306800

The WITBLO11 Tribute


A showcase of the 25 best levels of 2011, including such fantastic levels as Niveau Victorieux Galaxy, Plastic Beach, Dimension Infinity, Realm of the Heavens/Parallel Spires, and more!

My Level Collection: http://runouw.com/forums/viewtopic.php?f=25&t=47186

Other Important Links:
Last Legacy Level Designer tutorial
Eternal SM63 Medal Table
Eternal SM63 Score Table (all LDC scores available)
Top 10 Lists for LDC's (highest averages in a single judging criterion)
Runouw Votes Eternal Table
User avatar
Supershroom
Our local SM63 Spielberg

 
Posts: 1802
Joined: June 21st, 2014, 5:22 am
Location: Filmstudio for SM63. Sometimes also LL.

Runouw Votes Winner
New Years 2016: Best Level Winner (Ride Through The Mines)

Thumbs Up given: 244 times
Thumbs Up received: 126 times

Re: Request for Transparency

Postby ~MP3 Amplifier~ » May 12th, 2015, 8:33 am

Supershroom wrote:You've described perfectly what I've meant. Many deducts fall under the subjectivity that a judge has a right on, but negligence is avoidable as well as being too nit-picky when giving the 6/10 as mentioned.


Yeah, I mean the judge has a right to score it 6/10 if that's how they genuinely feel about the level.
But if the description in their judging says "great level, with just a few flaws" followed by "6/10", all we'd ask is that they either change the score to something fairer, or that they go into more depth about why they haven't scored it higher, if they do indeed have a reason to score it 6/10.

This is just an example but yeah.
"I treat everyone equally, depending on how much I like them." ~Me
The below image is a montage of my individual highest placing LDC creations, as a reminder to myself that level designing is a part of my life that I can't just leave behind

Image
I made this sig so credits to mee :amp smile:
User avatar
~MP3 Amplifier~
THE DARK LAMP

Error contacting Twitter
 
Posts: 4383
Joined: May 6th, 2011, 12:35 pm
Location: Maaars d(^_^)b

SM63 Level Designer Contest Winner
LDC #30: Finale/The Ultimate Battle

Thumbs Up given: 226 times
Thumbs Up received: 611 times

Re: Request for Transparency

Postby Harmless » May 12th, 2015, 7:36 pm

Last time I checked the judges did explain themselves though. -_-
Expect something cool here soon!

~ Tesla Bromonovich
User avatar
Harmless
Is it lunch time yet?

 
Posts: 2793
Joined: June 25th, 2011, 11:53 am
Location: Mother Russia!

Runouw Votes Winner
For winning Master of a Hidden Talent in the RV Summer 2017

Thumbs Up given: 271 times
Thumbs Up received: 240 times

Re: Request for Transparency

Postby nin10mode » May 12th, 2015, 8:22 pm

My counters to that are these:
Spoiler: show
Nin10mode
Omniquet
Fun- 7.5/10
This level was a lot of fun to trudge through at first, considering I'm a sucker for puzzles, but as the level became more mind-numbing, it started to lose its original friendly flair and come into true, raw, challenge. I love that.
Graphics- 3.5/5
The tilework is quite decorative and nicely planned, but not very much décor. Hm.
Other- 4/5
+1 no lag, good execution
FINAL SCORE: 15/20
Good 'n' challenging puzzle level, probably deserves a higher rating due to my lack of skill. :P

Fort Omniquet by nin10mode


Fun: 7/10 The level was a puzzle level. A way too hard puzzle level. Nothing more to say. It was fun, yes. But it was just too hard for me.


Once again I'll say as well, to counter-argue my own point: It's really easy to find patterns that aren't there if you actively look for them.

Graphics: 4.25/5 For a castle level, pretty good graphics. It didn't look like boring like most castle levels do. It looked good.

Other: 3.75/5 +0.5: No lag or loading time.
+0.1: Good ideas. Just too hard.
+0.15: Fake bridge was a really good idea.

Overall: 15/20

Nin10mode

Fun-10/10

I had a ton of fun in this level, but it wasn’t in the thrill ride I JUST OUTRUN THIRTY MOFOIN BULLET F’IN BILLS BEEEEECH kind of way. This was one of the most fun levels I’ve played, like, ever, because it’s the perfect balance of platformer, puzzle, easy, challenging, relaxing, invigorating, puzzling, clear- it’s really just the perfect balance of everything you want in a level, a very rare thing to ever accomplish in any gaming type thing. I’m proud of you, good sir.

Graphics-4.75/5
And damn me it looked great too. You always joke about the legendary 6/20 over your 7th LDC, but that was because things were too ruined. Again, balance here. I loved the eloquent tile blends here, and even if it was a bit off in some areas, it looked beautiful, to say the least, especially in the last room.

Other-4.25/5
-.5 There were avoidable instances of getting squished in the level in many areas.
+.75 Excellent, excellent music choice. *saves to library*. It actually was a good balance to the level, it kept me calm when otherwise I would not be.
+.5 I actually love that Runouw level you based the fire room off of. Good reference, and it looks just as great.
+.5 Great usage of disabled blocks for excellent mindtrickery.

Overall-19/20

If you want to do some math magic with their judging scales, here are the originals:
viewtopic.php?f=22&t=35020&hilit=omniquet&start=450#p207319
viewtopic.php?f=22&t=35020&hilit=omniquet&start=450#p207328
viewtopic.php?f=22&t=35020&hilit=omniquet&start=470#p207356

My argument here won't be "hurr I should have scored higher le"
Instead it will be why? Why does SuYo describe the enjoyment he had by saying "I love it," and give it a 7.5/10 rather than a 9 or 10? There's nothing negative listed.
And is difficulty really the only problem? What kind of audience should I be designing to? There must have been something positive to warrant a 7/10 from SMIC. What brought it up now that I vaguely recognize what's 'wrong' ?
If you look at Moy's, there are no contradictions.

That said, I'm not saying this is a trend, and I'm even being a bit hypocritical here with my light-hearted reviews (unless you really, REALLY know me), but there are certain measures that should be possible to prevent stuff like this.
Image
Art: show
Anime: show
Imagei use mal now but this sigbar is pretty
Videos: show
ImageImage
im@s: show
Image
User avatar
nin10mode
Immune to Death

Error contacting Twitter
Error contacting last.fm
 
Posts: 3087
Joined: October 3rd, 2009, 6:11 am
Location: Not here

Credit To Team

Thumbs Up given: 102 times
Thumbs Up received: 221 times

Re: Request for Transparency

Postby Harmless » May 12th, 2015, 8:48 pm

I did not make myself clear, sorry. I mean recent judgings have stepped up and started providing more explanations as to why something went wrong or right in a level, from their personal experience. Also from objective facts in the level, but either way, Fort Omniquet judgings were quite a while ago.

Heck, recent judgings are now WAY longer than the judges years ago, and that's because people are starting to explain themselves.
Expect something cool here soon!

~ Tesla Bromonovich
User avatar
Harmless
Is it lunch time yet?

 
Posts: 2793
Joined: June 25th, 2011, 11:53 am
Location: Mother Russia!

Runouw Votes Winner
For winning Master of a Hidden Talent in the RV Summer 2017

Thumbs Up given: 271 times
Thumbs Up received: 240 times

Re: Request for Transparency

Postby MessengerOfDreams » May 14th, 2015, 3:39 pm

In response to Nin10mode, that really shows the problem in some LDCs- organization. The 18th LDC was a mess. Star didn't calculate the results properly, 30 different judges did 30 different levels, people didn't show up, all kinds of messed up ♥♥♥♥. So more than anything, we need to make sure every LDC is organized perfectly.
Image
Image

My Most Recent Works: show
I switch my signature a lot. If you wanna see some of my past ones, here you go.
Silent Conversations and a Crow's Final Song!
My latest story, and one of my personal favorites. A girl bound in silence finds the words to say to her prospective girlfriend as they visit her religious father in a dusty town on the edge of Kansas, where the crows' migration south brings forth anchored memories, the path to resolution, and a new start.

Form (25quared)
This might be like nothing you've seen before.
Updated Works! Some of my past best and current stuff: show
Writing Works!
Fanfic: Shut Up and Dance
why do I write so much about dancing you don't dance you've never danced in your entire life
lying little ♥♥♥♥ with your ♥♥♥♥ story ♥♥♥♥ you
also Diddy/Lucina <3

Fanfic: Worth a Thousand Words
Because the world needed a Samus/Dedede story
Fanfic: Ecstatic Silence
Just wanted to write and ♥♥♥♥ like this happens, you'd think I'd know better.
Fanfic: Far From the Edge
It's a dance that's been a long time coming for a brand new man and an unchanging woman, but once one takes the plunge there's no falling back up.
Original: Jealous Ghosts of the Mississippi
The story of Rachel meeting Amber after a lifetime of silence and being shunned
Original: Your Hand in Mine
One of my most personal stories about a dangerous romance. Now to be published in a college lit journal!
-------------------------
Recent LDC Work:
Level Series: Leaves From The Vine (ft Star King)
For the 29th LDC, a theme of grassland taken through the growth of our designing society. Won the 29th LDC!
--
Levels at Large:
Level Topic: Collection of MoD's Levels!
If you've ever wanted to see any level I made worth a damn, go here! From the quiet 14th LDC entrant Finis to the megasmash level series Dark, you can find links here!
"You were always a revolutionary, now there's just less of a chance of you crying in the corner." ~Ridder
User avatar
MessengerOfDreams
Moderator

Error contacting Twitter
 
Posts: 6615
Joined: August 16th, 2009, 11:31 am
Location: When I figure it out, I'll let you know.

Winter
2016 Story Contest

Thumbs Up given: 519 times
Thumbs Up received: 707 times

PreviousNext

Return to Super Mario 63 Level Designer