Request for Transparency

Level Designer Contests, discussion about and help with the Level Designer, information about glitches, and levels and level series can be found here.

Moderator: Level Moderator

Final judge discussion?

Yes
10
63%
No
6
38%
 
Total votes : 16

Re: Request for Transparency

Postby SuperMIC » May 27th, 2015, 8:03 pm

I think there's a little bit of miscommunication going on here: What Yuri's trying to say (correct me if I'm wrong bae) is that if a situation occurs such that an outlier score exists, then that judge just needs to talk with the others about why and provide a strong reasoning for the score. Which is still what Doram and Ven are saying: if it's justifiable, fine, but outlier points have a potential to be caused by some faulty reasoning.

However, there is something that I have to agree with MoD on here: and that is that "I didn't have fun" is indeed a valid reason to give a score. Obviously, the judge would need copious details on why they didn't have fun: breaking down their experiences with specific level sections, etc. But again, the category is literally called "Fun" for a reason. The point of a game is for it to be actually fun. To say "I didn't enjoy this, but it looks like I should enjoy this" is not solid reasoning, and in fact, the discussion would cause that judge to lower their points (yes this happened during our discussion, sorry fellow peoples yes the discussion can cause judges to lower scores as well).
<Arocado:> I'm Asian.
PROOF.

Image
^ Panik, you rock. ^

RUNOUW: DANGER AND DECEPTION: Read it! You could be in it!

"Surveys": show
Image
Image

Image
User avatar
SuperMIC
The Legacy

 
Posts: 692
Joined: February 2nd, 2011, 12:08 pm
Location: Br... it's chilly here, eh?

Thumbs Up given: 85 times
Thumbs Up received: 60 times

Re: Request for Transparency

Postby ~Yuri » May 27th, 2015, 8:14 pm

SuperMIC wrote:I think there's a little bit of miscommunication going on here: What Yuri's trying to say (correct me if I'm wrong bae) is that if a situation occurs such that an outlier score exists, then that judge just needs to talk with the others about why and provide a strong reasoning for the score. Which is still what Doram and Ven are saying: if it's justifiable, fine, but outlier points have a potential to be caused by some faulty reasoning.


Yup, that's exactly what I was trying to say (ty <3).

SuperMIC wrote:However, there is something that I have to agree with MoD on here: and that is that "I didn't have fun" is indeed a valid reason to give a score. Obviously, the judge would need copious details on why they didn't have fun: breaking down their experiences with specific level sections, etc. But again, the category is literally called "Fun" for a reason. The point of a game is for it to be actually fun. To say "I didn't enjoy this, but it looks like I should enjoy this" is not solid reasoning, and in fact, the discussion would cause that judge to lower their points (yes this happened during our discussion, sorry fellow peoples yes the discussion can cause judges to lower scores as well).

The category's name has changed for that exact same reason actually. It's not "Fun" anymore, it's "Gameplay", because the judge should be telling what he thinks about the gameplay. If there's any kind of innovative gameplay, good execution of a concept, and of course, if the level is fun should be counted on this category.
Image
credits to Nan for this amazing gif ^-^
other sigs: show
Image
Awesome sig made by MessengerOfDreams, way back in 2013. She's awesome <3
Image
Amazing sigbar made by FrozenFire!
Image
Image

Great sig made by Raiyuuni!

Image

Made by Konradix! ^-^ (clickable!)
User avatar
~Yuri
Boo

 
Posts: 677
Joined: December 5th, 2012, 3:09 pm
Location: lost, can you please give me some directions?

Cookie
NanTheDark: "Congrats on winning our duel. May the trees watch over you."

Thumbs Up given: 46 times
Thumbs Up received: 85 times

Re: Request for Transparency

Postby MessengerOfDreams » May 27th, 2015, 8:22 pm

That was an official change? I still call it fun. And again you seem to talk like there's a gameplay criteria we are required to meet the standards of.
Image
Image

My Most Recent Works: show
I switch my signature a lot. If you wanna see some of my past ones, here you go.
Silent Conversations and a Crow's Final Song!
My latest story, and one of my personal favorites. A girl bound in silence finds the words to say to her prospective girlfriend as they visit her religious father in a dusty town on the edge of Kansas, where the crows' migration south brings forth anchored memories, the path to resolution, and a new start.

Form (25quared)
This might be like nothing you've seen before.
Updated Works! Some of my past best and current stuff: show
Writing Works!
Fanfic: Shut Up and Dance
why do I write so much about dancing you don't dance you've never danced in your entire life
lying little ♥♥♥♥ with your ♥♥♥♥ story ♥♥♥♥ you
also Diddy/Lucina <3

Fanfic: Worth a Thousand Words
Because the world needed a Samus/Dedede story
Fanfic: Ecstatic Silence
Just wanted to write and ♥♥♥♥ like this happens, you'd think I'd know better.
Fanfic: Far From the Edge
It's a dance that's been a long time coming for a brand new man and an unchanging woman, but once one takes the plunge there's no falling back up.
Original: Jealous Ghosts of the Mississippi
The story of Rachel meeting Amber after a lifetime of silence and being shunned
Original: Your Hand in Mine
One of my most personal stories about a dangerous romance. Now to be published in a college lit journal!
-------------------------
Recent LDC Work:
Level Series: Leaves From The Vine (ft Star King)
For the 29th LDC, a theme of grassland taken through the growth of our designing society. Won the 29th LDC!
--
Levels at Large:
Level Topic: Collection of MoD's Levels!
If you've ever wanted to see any level I made worth a damn, go here! From the quiet 14th LDC entrant Finis to the megasmash level series Dark, you can find links here!
"You were always a revolutionary, now there's just less of a chance of you crying in the corner." ~Ridder
User avatar
MessengerOfDreams
Moderator

Error contacting Twitter
 
Posts: 6615
Joined: August 16th, 2009, 11:31 am
Location: When I figure it out, I'll let you know.

Winter
2016 Story Contest

Thumbs Up given: 519 times
Thumbs Up received: 707 times

Re: Request for Transparency

Postby ~Yuri » May 27th, 2015, 8:33 pm

Yeah, we talked about that on the LeMod forums before, and everybody supported the new names. So yeah, it's an official change yup.

And what I was trying to say is that you should not only consider your personal fun now. You should consider the difficulty, the innovation in the gameplay, the execution, and again, fun.
Image
credits to Nan for this amazing gif ^-^
other sigs: show
Image
Awesome sig made by MessengerOfDreams, way back in 2013. She's awesome <3
Image
Amazing sigbar made by FrozenFire!
Image
Image

Great sig made by Raiyuuni!

Image

Made by Konradix! ^-^ (clickable!)
User avatar
~Yuri
Boo

 
Posts: 677
Joined: December 5th, 2012, 3:09 pm
Location: lost, can you please give me some directions?

Cookie
NanTheDark: "Congrats on winning our duel. May the trees watch over you."

Thumbs Up given: 46 times
Thumbs Up received: 85 times

Re: Request for Transparency

Postby -BY » May 27th, 2015, 9:04 pm

The Fun/Gameplay section is indeed not just one thing. It can and should be split up into it's different sections. Afterwards it can of course get summarized again.
While reading through the most recent posts, I've kind of a feel that there are people who actually missed the part that the "Fun" score is more than just one single thing, but is rather a collection of those. Furthermore I personally see my general feel of it as a part of said collection rather than the whole thing in general. Now I'm like wondering if this whole misunderstanding originates from the sole definition of the gameplay/fun score actually.
Image

Big thank you to FrozenFire who created this masterpiece. : )
User avatar
-BY
Honorary Member

 
Posts: 900
Joined: May 23rd, 2012, 11:43 pm
Location: 298

Razzian Fighter

Thumbs Up given: 72 times
Thumbs Up received: 100 times

Re: Request for Transparency

Postby Harmless » May 27th, 2015, 9:13 pm

I always assumed 'fun' was gameplay actually.
Expect something cool here soon!

~ Tesla Bromonovich
User avatar
Harmless
Is it lunch time yet?

 
Posts: 2793
Joined: June 25th, 2011, 11:53 am
Location: Mother Russia!

Runouw Votes Winner
For winning Master of a Hidden Talent in the RV Summer 2017

Thumbs Up given: 271 times
Thumbs Up received: 240 times

Re: Request for Transparency

Postby -BY » May 27th, 2015, 9:15 pm

Inb4 we need to get down to the roots and explain what every aspect actually is supposed to cover.
Image

Big thank you to FrozenFire who created this masterpiece. : )
User avatar
-BY
Honorary Member

 
Posts: 900
Joined: May 23rd, 2012, 11:43 pm
Location: 298

Razzian Fighter

Thumbs Up given: 72 times
Thumbs Up received: 100 times

Re: Request for Transparency

Postby Supershroom » May 28th, 2015, 2:40 am

This entire "blowout" solely originated from me just asking about how things go, and then certain people suddenly feeling the need of destructively provoking and to get their own back on me while I've already made the concessions the staff and others wanted to hear all the time. Some people (including me) tried to mediate and hold a constructive discussion, but others kept jumping on that train of venting all their ressentiments and aggressivity, instead of giving me a chance to keep my promises.

And to those who say "everything justifiable is alright". You're all lacking a concrete definition of "what is justifying", because in general it's not possible to justify scores simply by your own stamp of infallibility or something like that.

Also Doram, I seriously disapprove of your accusation of my complaints about "unfairness" (or what appeared to be unfair to me) being an INSULT, and of your reasoning, "the judge tried to be as fair as possible". He is still fallibe. Referees also try to be as fair and as unbiased as possible. And still they make mistakes, sometimes even very embarassing mistakes. There is the thing about the so called "factual decision", but everyone supporting the irrevocability of judgings / a ref's decision forgets that judges / refs not only decide facts, they also create facts.

I'll say it again if it's necessary, quit your ressentiments.

I actually think that if you're reading through the Judging style topic, it's already pretty much clear what's to be covered in Gameplay and Graphics. There may just be some explanation needs on the Other tier, as judges actually have very diversified ranges of plus / minus points, and also sometimes put things in Other that are already covered else-where. I'd maybe give an extra deduct for lots of cutoff or enemy spam, as well as a +0.5 for extremely well done gameplay mechanics or a very beautiful tiling / decoration scheme, but I'd never dock off a -0.5 saying "The level is far too hard" as I've already covered that in Gameplay. Some more unification could help to prevent that one judge gives 4/5 in Other and another only gives 1/5.
Image

Thank you, FF.

Nan's sig! From the previous Secret Santa event!: show
Rating standards: show
1 star: (0 - 5.75 in LDC) These levels need a lot of work. They're totally un-elaborated, and/or inherently flawed due to things like cutoff or enemy spam, or they're untested and there are too many bugs gameplay-wise, in short: Many many things went wrong in such a level.

2 stars: (6 - 9.75) These levels are not terrible, but poor. They're too short, lack scenery or they have errors and/or bugs but it's still barely enjoyable. Again, spend more effort and try to do better. (if it's your first level that I rate two stars, it's not bad. There's a long way you can still go)

3 stars: (10 - 12.75) These levels are about mediocre. They may be still a little bit short, the gameplay is fine but not very original and graphics are also solid, but not breath-taking. Try to keep improving!

4 stars: (13 - 15) These levels are nice, but not the best. Effort has been put into them, there can some professionality be seen in gameplay and graphics, they're a nice experience to play, but there are still things you can do to make it even better. Try to go the extra mile!

5 stars: (15.25 - 20) These are really great levels, those which, as said, I would give more than 15 in an LDC. These suffice many high requirements, and it already goes into subtleties if you want to make them better. These levels are successful all around. Bravo.

These are just general principles I try to follow when rating. Sometimes I don't leave a further comment when rating, most times I do, if you have questions about it, feel free to ask further.
Look at this!: show
My total SM63 level experience: 2803 levels.
Total coins collected in these: 306800

The WITBLO11 Tribute


A showcase of the 25 best levels of 2011, including such fantastic levels as Niveau Victorieux Galaxy, Plastic Beach, Dimension Infinity, Realm of the Heavens/Parallel Spires, and more!

My Level Collection: http://runouw.com/forums/viewtopic.php?f=25&t=47186

Other Important Links:
Last Legacy Level Designer tutorial
Eternal SM63 Medal Table
Eternal SM63 Score Table (all LDC scores available)
Top 10 Lists for LDC's (highest averages in a single judging criterion)
Runouw Votes Eternal Table
User avatar
Supershroom
Our local SM63 Spielberg

 
Posts: 1802
Joined: June 21st, 2014, 5:22 am
Location: Filmstudio for SM63. Sometimes also LL.

Runouw Votes Winner
New Years 2016: Best Level Winner (Ride Through The Mines)

Thumbs Up given: 244 times
Thumbs Up received: 126 times

Re: Request for Transparency

Thumbs up x1

Postby Asterocrat » May 28th, 2015, 6:35 am

You see Shroom, the problem with your post right here is that you are comparing Level Designing Contests to a sport event, and judges to referees. By that, you are admitting that there is some sort of general guidelines, like lines on the ground and nets at both ends of the court, as how to actually judge a level. Truth is, there is not. To judge a level, there is no clear line on the ground to tell you if the level is right or is wrong for a very simple reason: It's going to sound cheesy, but it's the downright truth: You don't judge a level with your reasoning and counting how many coins there are in a level, but with your heart, with your feelings. You base your scores based on how you felt after playing level, based on the different emotions of joy, frustration or awe you encountered when playing the level, therefore a level is more comparable to a cultural piece (artwork, movie) than a sport match. Therefore, there is no way there is such a thing as a "mistake" when judging a level. Here, let me use an example that is surprising close to Level Designing Contests to illustrate.

My example is Dark Souls. For those who don't know, Dark Souls is a exploration/adventure video game that supposedly is extremely difficult, with unforgiving parts, almost unfair gameplay and extremely rare save points. Jeez, that sounds rather similar to some levels we had to judge in the past, doesn't it? Anyway, to cite an example, probably the most crucial item in the game, necessary to advance, is hidden very deep in the game and the game doesn't give you ANY clue as how to obtain it. Opinions about the game are divided into two groups: one side blames the game for its obviously frustrating gameplay, while the other side claims it to be one of the greatest games because this sense of helplessness is one of the most ingenious game design in any video game. Obviously opinions feel rather different, even complete opposites, but does that necessarily mean that one of the sides is wrong and the other one is right? No. Because everyone's opinion on the game is based on what they enjoy in a video game, their opinion on game design in general, it's how they are, it's how they feel, and because of that they can't be wrong.

Now, let's say we take a sample from everyone that can have an opinion on Dark Souls. Let's say, uh... 5 people, we'll call them judges. Four of them happen to dislike the game entirely, they all give it an average score because it plays well and have decent graphics, but it's still very frustrating. The last one just happens to love the game, because it's exactly what he expected to see in a video game, he gives it a pretty high score. Now, because we only have a smaller sample, because that one guy rated much higher than the others, does that mean he'll have to consider his thought process, question his feelings about the game, reconsider his judging? No! We (hopefully) agreed up there that his opinion was a valid opinion. It's an opinion like any other, but he just happened to be the only one who felt that way about the game in the given sample. Does that mean is opinion is NOW wrong? Still no, because that would be concerning.

What I'm trying to say is, as different as it is from the "pseudo-norm" you guys are defining, an opinion, a score, cannot be wrong because it's more based on our emotions, than on actual facts. I'm not saying that there are no rules a level should follow (for example Dark Souls is still beatable, similarly, a level should be beatable), but a difference, even a drastic one, is more than normal when judging because, as I said in another post, we are only human.

With all that in mind, I think judging should continue as it is right now; however, I think it should be a good idea that the host check the scores before posting the results. If a score is way too different from the average from the other judges (say, 5-point difference) on a specific level, then he should check the justifications for this score. If then the host is not satisfied with the justification, he could have the right (if you guys still think we need it) to ask the judge to either reconsider his judging, or give more justifications. However, as proven by the current 25quared LDC, a systematic process (with the meeting of the judges) in ALL LDCs with ALL judges is most of the times unnecessary and hard to plan (getting all judges online at the same time is pretty hard).

Just my honest and non-aggressive two cents.
Image
Gift by Forgotten - Full size here!: show
Image

Image

User avatar
Asterocrat
Former Cosmic Ruler

Error contacting Twitter
 
Posts: 2243
Joined: July 1st, 2010, 2:37 am
Location: Strasbourg, Alsace, Eastern France

Runouw Votes Winner
Best Super Mario 63 Level (Can't fix what isn't broken) with "Neon City" in Runouw Votes Summer 2014!

Thumbs Up given: 266 times
Thumbs Up received: 309 times

Re: Request for Transparency

Thumbs up x1

Postby Konradix » May 28th, 2015, 6:39 am

Shroom. On paper, judges and referees seem to be the same. However, they're two completely different roles. Referees role is to make sure that everyone follows the rules, clear instructions of how you play a game. If you break the rules, it's his job to call you out, as you broke the rules, making the game unfair for your team. While I'm not gonna imply that the job is simple, his role is founded on something that is either black or white. You follow the rules, or you break them. There is no middle ground.

On the other hand a Judge is someone who will review and rate someone's work based on their preference and opinion. While there obviously are rules of what you should praise and what you shouldn't (like deducting points for no 1UPs in the level), there is also a large gray area in between the "right" and "wrong". There are no clear cut instructions on what is a perfect level, or a bad one. What appeals to one person may not appeal to the other. One, two, million opinion's of judges does not make that opinion fact and never will.

An example: What music do you listen to? I listen to Rock mostly. However in almost all cases, I cannot stand Dubstep. It does not sound appealing and I do not like the weird noises it makes. I also do not enjoy rap. I feel like it's kind of pretentious talking. Or Pop because... I dunno. I just don't like it.

Now tell me, are these justifications invalid because another group of people likes rap/pop/dubstep? Is my opinion wrong because my opinion is different to that group of people? Is their opinion wrong because their opinion differs?

How about another example. Try and rate a piece of abstract art. Like this. To many people it may look very appealing and interesting. To another these will look like someone sneezed on a canvas and sold it for overly large price. It is not interesing or good. Is his opinion wrong because he is the single outlier to the trend?

Referees rely on facts that are listed in the book. Judges do not. And while rules for things like football may be amended, it still doesn't change the fact that their role is to call out the wrong. A Judge cannot do that, since their judging is by nature subjective. There are no clear 'rules' for a perfect level. Perfection is different to everyone.

@EDIT: Also :ninja: by StarK. He basically said what I said, except you don't need to decrypt his ♥♥♥♥.

StarEDIT: Haha, sorry. Funny how we said very similar things though
"Men cry not for themselves, but for their comrades"

Check out my newest level called:
Image
Be sure to leave a comment and a rating ;)
Fun and Games: show
Levels that I may make by 20XX
- Remake of Airship Armada
- Level using the Turbo FLUDD
- Level using the Flying Cap
- A level on the moon

- Syzygy Labs
- Maybe some sort of Metroidvania level
- Sneaking level
- A mystery


Random Stuff
Image
Image
Czasoprzestrzen/Spacetime... If only somebody knew this hidden gem...

Huh?
GENERATION 31: The first time you see this, copy it into your sig on any forum and add 1 to the generation. Social experiment.
User avatar
Konradix
The Legacy

 
Posts: 689
Joined: May 12th, 2010, 8:14 am
Location: Studying the Grand History in the Ruins of the World

Runolympics 2015 Participation

Thumbs Up given: 85 times
Thumbs Up received: 39 times

PreviousNext

Return to Super Mario 63 Level Designer