Overhaul of SM63 judging system

General discussion about the portal. The Level Posting Rules are also here.

Moderator: Level Moderator

The final concept has been made. Do you want it?

Poll ended at May 5th, 2016, 3:06 am

Yeah, I'm down with a 2-category system and I think it will change things for the better.
14
74%
No, I want to stick to the current system.
5
26%
I'm not really content with both of them. (express your comments further)
0
No votes
 
Total votes : 19

Re: Overhaul of SM63 judging system

Postby Supershroom » April 30th, 2016, 2:48 am

Okay guys, since we've gone off-topic and gone back to the old discussion if you can properly do story with the SM63 designer - you CAN, but gameplay still shouldn't get the short end of the stick. That's what e.g. Timpani & Blumiere or Tearing Paper have managed to do well. If you mix gameplay mechanics and story to a homogeneous symbiosis, it's a totally different experience than just reading the story and listening to the music itself. A good judge may not be a fan of story or puzzle by personal taste, yet he can evaluate how well-executed the story is and how well gameplay is done, and judge that accordingly.

Now, first off, if I really followed egoistic motives with this, I should rather be interested in 10/10 because I usually tend to struggling with difficulty and bugs, but getting high Graphics scores. But I don't wish for cheap laurel. The reason why Other should be banned from the planet, and why there should be guidelines on proportions of certain aspects, is that we all have our own little idiosyncrasies, and not because of disadvantaging / depreciating a certain designing style. Things like "1.5 points for music" or "check the causes of loading time / lag" are recommendations, you can do it slightly differently if you feel like, but if you do it too extremely, an LDC host should be there to step in.

Now, why (usually) 12/8 and not 10/10? I've said it from the very beginning already - Other used to contain both gameplay aspects and artistic / atmospheric aspects in approximately equal shares. And gameplay always was worth twice as much as Graphics. In fact it should be even 12.5/7.5, and simply for convenience reason, I suggest 12/8 (alias 15/10), it does the best at keeping things mostly the same, but if we move to 10/10 and have bugs, innovations etc. in the first half, we only create a bias towards a direction we've never moved at before. Again, no rule without exception though, we could always use 10/10 on a theme which focuses on story and music (and these two would be upgraded, while Graphics should keep being worth about 5 points).
Image

Thank you, FF.

Nan's sig! From the previous Secret Santa event!: show
Rating standards: show
1 star: (0 - 5.75 in LDC) These levels need a lot of work. They're totally un-elaborated, and/or inherently flawed due to things like cutoff or enemy spam, or they're untested and there are too many bugs gameplay-wise, in short: Many many things went wrong in such a level.

2 stars: (6 - 9.75) These levels are not terrible, but poor. They're too short, lack scenery or they have errors and/or bugs but it's still barely enjoyable. Again, spend more effort and try to do better. (if it's your first level that I rate two stars, it's not bad. There's a long way you can still go)

3 stars: (10 - 12.75) These levels are about mediocre. They may be still a little bit short, the gameplay is fine but not very original and graphics are also solid, but not breath-taking. Try to keep improving!

4 stars: (13 - 15) These levels are nice, but not the best. Effort has been put into them, there can some professionality be seen in gameplay and graphics, they're a nice experience to play, but there are still things you can do to make it even better. Try to go the extra mile!

5 stars: (15.25 - 20) These are really great levels, those which, as said, I would give more than 15 in an LDC. These suffice many high requirements, and it already goes into subtleties if you want to make them better. These levels are successful all around. Bravo.

These are just general principles I try to follow when rating. Sometimes I don't leave a further comment when rating, most times I do, if you have questions about it, feel free to ask further.
Look at this!: show
My total SM63 level experience: 2803 levels.
Total coins collected in these: 306800

The WITBLO11 Tribute


A showcase of the 25 best levels of 2011, including such fantastic levels as Niveau Victorieux Galaxy, Plastic Beach, Dimension Infinity, Realm of the Heavens/Parallel Spires, and more!

My Level Collection: http://runouw.com/forums/viewtopic.php?f=25&t=47186

Other Important Links:
Last Legacy Level Designer tutorial
Eternal SM63 Medal Table
Eternal SM63 Score Table (all LDC scores available)
Top 10 Lists for LDC's (highest averages in a single judging criterion)
Runouw Votes Eternal Table
User avatar
Supershroom
Our local SM63 Spielberg

 
Posts: 1802
Joined: June 21st, 2014, 5:22 am
Location: Filmstudio for SM63. Sometimes also LL.

Runouw Votes Winner
New Years 2016: Best Level Winner (Ride Through The Mines)

Thumbs Up given: 244 times
Thumbs Up received: 126 times

Re: Overhaul of SM63 judging system

Postby MessengerOfDreams » April 30th, 2016, 6:25 am

50% was never the short end of the stick and giving it unbalanced priority is definitely not. In fact it's giving everything else the short end of the stick. I don't get how anyone doesn't see the logic of that.
Image
Image

My Most Recent Works: show
I switch my signature a lot. If you wanna see some of my past ones, here you go.
Silent Conversations and a Crow's Final Song!
My latest story, and one of my personal favorites. A girl bound in silence finds the words to say to her prospective girlfriend as they visit her religious father in a dusty town on the edge of Kansas, where the crows' migration south brings forth anchored memories, the path to resolution, and a new start.

Form (25quared)
This might be like nothing you've seen before.
Updated Works! Some of my past best and current stuff: show
Writing Works!
Fanfic: Shut Up and Dance
why do I write so much about dancing you don't dance you've never danced in your entire life
lying little ♥♥♥♥ with your ♥♥♥♥ story ♥♥♥♥ you
also Diddy/Lucina <3

Fanfic: Worth a Thousand Words
Because the world needed a Samus/Dedede story
Fanfic: Ecstatic Silence
Just wanted to write and ♥♥♥♥ like this happens, you'd think I'd know better.
Fanfic: Far From the Edge
It's a dance that's been a long time coming for a brand new man and an unchanging woman, but once one takes the plunge there's no falling back up.
Original: Jealous Ghosts of the Mississippi
The story of Rachel meeting Amber after a lifetime of silence and being shunned
Original: Your Hand in Mine
One of my most personal stories about a dangerous romance. Now to be published in a college lit journal!
-------------------------
Recent LDC Work:
Level Series: Leaves From The Vine (ft Star King)
For the 29th LDC, a theme of grassland taken through the growth of our designing society. Won the 29th LDC!
--
Levels at Large:
Level Topic: Collection of MoD's Levels!
If you've ever wanted to see any level I made worth a damn, go here! From the quiet 14th LDC entrant Finis to the megasmash level series Dark, you can find links here!
"You were always a revolutionary, now there's just less of a chance of you crying in the corner." ~Ridder
User avatar
MessengerOfDreams
Moderator

Error contacting Twitter
 
Posts: 6615
Joined: August 16th, 2009, 11:31 am
Location: When I figure it out, I'll let you know.

Winter
2016 Story Contest

Thumbs Up given: 519 times
Thumbs Up received: 707 times

Re: Overhaul of SM63 judging system

Thumbs up x1

Postby npromin1 » April 30th, 2016, 7:08 am

I think it's time to change the poll to be 10/10 vs 12/8 because the posts seem to have settled into discussion about that, rather than the two-category system. Based on the votes, most of us are good with the 2-category system.
npromin1
As Ninja As Myst

 
Posts: 167
Joined: July 20th, 2015, 4:11 pm

Thumbs Up given: 28 times
Thumbs Up received: 7 times

Re: Overhaul of SM63 judging system

Postby NanTheDark » April 30th, 2016, 7:44 am

I think we need to properly what "gameplay" means. If we were to judge two different games, each game has different gameplay. However, all SM63 levels will have a Mario that can walk, run, jump, triple jump, dive... So when we say gameplay we can't really mean how the level is played since all levels operate under the same basic mechanics. So in gameplay we would probably judge raw level design. Placing of platforms, items, enemies, interesting setpieces, the level's "flow" or "momentum", puzzles and other interesting platforming concepts. This is what you push buttons to go through. Interactivity is, in my opinion, the most important thing in a videogame. That's what separates it from watching a movie, reading a book, and other forms of fiction.

That's why I believe that gameplay should be above thinks like graphics, or some "alternate music" or a sign with a lot of text in it. The most extreme example I can think of a level which is nothing but story and graphics is a flat ground with a lot of hills and trees and cool looking things and a lot of signs. If you really wanted to JUST make a beautiful landscape, make a drawing. If you really wanted to JUST tell a story through text, write a novel.

I am not saying that story is bad, that graphics are bad. I am not saying that a great level should just be the Champion's Road (and also, I'm against frustratingly hard levels where it seems that something is hard not because the player is making a mistake, but because the designer made bad design choices. I am not going to name anyone on this, you know who you are). What I'm saying is that people who like storytelling, like MoD and the beautiful, amazing, great singer and best girl ever MP3 should take advantage of the fact that this is a game. What you should do, is tell story through gameplay.

Let's assume for a moment that your levels were like my extreme example of just flat ground (which they're not, I'm sure of it, I'm just trying to make a point). If you wanted to make a story about a hero traveling the land, then design the trip itself. Make us feel the journey, instead of putting a sign that says "I've gone in a great journey!". Make us play it. If you want to represent someone's inner mental state, you design some trippy looking landscape that represents the person's twisted mind, instead of making a sign that says "I am a very depressed person".

Show,don't tell.

If you think that putting more points into gameplay means your story will be shunned or biased against, all I have to say is that maybe you're not making the most of this particular media in order to tell a story. You can either make us sit through cutscenes (signs) in order for us to know what the plot is, or you can make us play the story itself, as if we were actors. You can make us figure things ourselves by the look of the terrain, a long climb, taking a leap of faith over an abyss.

And then, your story isn't just Atmospherics. It is also Gameplay.

Also MoD, you can always just post levels in the portal if you feel that we are limiting storytelling, which we are not. Maybe you just don't know how to properly tell a story through an interactive media (which I can't say for sure, I haven't really played too many of your levels. I'm assuming a extreme case).

Also stop thinking that everyone's out to get you. You can pretend to be a victim as much as you want, but everyone here loves you and thinks you're one of the greatest LDers ever.

That is mostly my argument as to why I think 12/8 is alright and why I think it shouldn't be a disadvantage for storytellers, but instead encouragement to make the most out of the media they're trying to tell a story in. And it is possible to tell a story in the way I suggest, there are games out there that do that. Stories that are about what the player is doing, or about exploring a metaphor, or whatever. You can do it.


















So yeah, that. *raises Flame Shield*
Some day I'll fix my sig.

Old Sig Stuff: show
Image

Universal Dueling ArenaInto The ShadowsMy SM63 Levels

Image
Made by FrozenFire/lemontea/whatever. Thanks, FF!

ImageImageImageImageImage
????
Image
User avatar
NanTheDark
As it breaks, it becomes stronger

 
Posts: 2018
Joined: December 23rd, 2010, 6:04 pm
Location: Your screen

Runouw Votes Winner
For winning Best Normal Member in the RV Summer 2017

Thumbs Up given: 191 times
Thumbs Up received: 217 times

Re: Overhaul of SM63 judging system

Postby npromin1 » April 30th, 2016, 8:42 am

I don't see much sense of adding plot to a SM63 level. (No offence,MoD)This isn't as good of a medium as LL is. The atmosphere in LL levels benefits more from plot,then SM63. Adding music and graphics to a SM63 level improves the quality of the level, but plot is optional. Therefore, I think that atmosphere in a SM63 level should mostly come from graphics and music.

If there's story in a level and it's executed well, those two points can serve as room to add points for atmosphere. If it's executed poorly, a deduction of points should be allowed. I'm not saying that those 2 points should be there for story. It should be possible to get a perfect score on atmosphere if the level is missing a plot-line, but if the level doesn't have a perfect score and there is a well-executed plot-line, then points should be added by the judge.

I don't know about everyone's designing skills, but it's un-fair to designers who are better with adding plot, music, and graphics to their level. I should be pushing for 12/8, since I tend to go too far with music, plot, and graphics and mess up instead. (My 5th 25quared level suffered greatly from that) 10/10 is the way to go.


Also you can't have good gameplay without a good atmosphere. It's that simple.


It's time to move on from this discussion and have the polls change to 10/10 VS 12/8 to get the general public opinion.
npromin1
As Ninja As Myst

 
Posts: 167
Joined: July 20th, 2015, 4:11 pm

Thumbs Up given: 28 times
Thumbs Up received: 7 times

Re: Overhaul of SM63 judging system

Postby Supershroom » April 30th, 2016, 8:56 am

As much as I see that the majority for a 2-category system in general is large, I'm not doing something premature already if I don't get permission from a few others.

npromin, I don't think you should let your experiences at the mini-LDC rule your opinion that much. Overall, it got mixed opinions pertaining your graphics. Also check out some series like Dark, Tearing Paper, The Blind Experiment orTimpani & Blumiere, all of them being in WITBLO12.

npromin1 wrote:If there's story in a level and it's executed well, those two points can serve as room to add points for atmosphere. If it's executed poorly, a deduction of points should be allowed. I'm not saying that those 2 points should be there for story. It should be possible to get a perfect score on atmosphere if the level is missing a plot-line, but if the level doesn't have a perfect score and there is a well-executed plot-line, then points should be added by the judge.

Yeah, you've worded pretty well how I imagine it to work myself.

npromin1 wrote:Also you can't have good gameplay without a good atmosphere. It's that simple.

Usually correct in the sense that good atmosphere automatically boosts gameplay a bit (which is another reason for 12/8 methinks), yet e.g. Volkove managed to be a god with gameplay and often enough he didn't need super-elaborated graphics to shine with that.
Image

Thank you, FF.

Nan's sig! From the previous Secret Santa event!: show
Rating standards: show
1 star: (0 - 5.75 in LDC) These levels need a lot of work. They're totally un-elaborated, and/or inherently flawed due to things like cutoff or enemy spam, or they're untested and there are too many bugs gameplay-wise, in short: Many many things went wrong in such a level.

2 stars: (6 - 9.75) These levels are not terrible, but poor. They're too short, lack scenery or they have errors and/or bugs but it's still barely enjoyable. Again, spend more effort and try to do better. (if it's your first level that I rate two stars, it's not bad. There's a long way you can still go)

3 stars: (10 - 12.75) These levels are about mediocre. They may be still a little bit short, the gameplay is fine but not very original and graphics are also solid, but not breath-taking. Try to keep improving!

4 stars: (13 - 15) These levels are nice, but not the best. Effort has been put into them, there can some professionality be seen in gameplay and graphics, they're a nice experience to play, but there are still things you can do to make it even better. Try to go the extra mile!

5 stars: (15.25 - 20) These are really great levels, those which, as said, I would give more than 15 in an LDC. These suffice many high requirements, and it already goes into subtleties if you want to make them better. These levels are successful all around. Bravo.

These are just general principles I try to follow when rating. Sometimes I don't leave a further comment when rating, most times I do, if you have questions about it, feel free to ask further.
Look at this!: show
My total SM63 level experience: 2803 levels.
Total coins collected in these: 306800

The WITBLO11 Tribute


A showcase of the 25 best levels of 2011, including such fantastic levels as Niveau Victorieux Galaxy, Plastic Beach, Dimension Infinity, Realm of the Heavens/Parallel Spires, and more!

My Level Collection: http://runouw.com/forums/viewtopic.php?f=25&t=47186

Other Important Links:
Last Legacy Level Designer tutorial
Eternal SM63 Medal Table
Eternal SM63 Score Table (all LDC scores available)
Top 10 Lists for LDC's (highest averages in a single judging criterion)
Runouw Votes Eternal Table
User avatar
Supershroom
Our local SM63 Spielberg

 
Posts: 1802
Joined: June 21st, 2014, 5:22 am
Location: Filmstudio for SM63. Sometimes also LL.

Runouw Votes Winner
New Years 2016: Best Level Winner (Ride Through The Mines)

Thumbs Up given: 244 times
Thumbs Up received: 126 times

Re: Overhaul of SM63 judging system

Postby Yoshi Boo 118 » April 30th, 2016, 10:10 am

Just to throw my two cents in here...I think for the most part, which 2-category system people go with really shouldn't matter. I'd probably go with the 10/10, but let's be realistic here. When it comes to judging levels for contests, it comes down to opinion. There have been times where I start judging one level for a contest, then I judge another level that gets the same score. But then I think to myself, "well this 2nd level really seems better overall in my opinion," so then I'll go back and either reduce a small number of points (ranging from .25 to maybe even a point) from the 1st or add to the 2nd. I think that's perfectly fair to do, because I believe that you may have a certain bias going (especially after not judging for a while) in that steadily disappears as you're judging. Even then, giving everyone the same score doesn't really say anything about how you feel about the levels comparatively.

What I'm saying is that I think in the end, each judge will give an overall score of what they feel the level deserves, and adjust the category scores accordingly. It doesn't feel right for any judge to give a score they think the level won't deserve. So I think 12/8 is doable, but it may skew the results a little more if judges have the same thought process that I have. I'd prefer 10/10.

So in conclusion,

nin10mode wrote:I'd just like to say that I only agree with the format as a means of organizing the old one in a better way. Any restrictions to the actual judging I don't agree with.
ImageImage

Awesome Things, and My Works: show
ImageImage

Credit to Buff for my normal avatar and lordpat for the Red Yoshi sig. Ninny made the Valentine's Gift. <3 Thanks!


Click on the left and right side of my main signature to see my Level Series (Paper Mario 63) and my Levels!

Stories (That are more than just a prologue long):
Unlocked (One of my first stories, pretty much abandoned now)
Koopa's Final Blow (Halloween 2011)
Resonance (Posted here first, but other link is better)
The Gibus Games (TF2 related)
My fanfics are posted here, where the most recent updates will be on.

Image
User avatar
Yoshi Boo 118
Ex umbra in solem

 
Posts: 1700
Joined: October 17th, 2009, 4:05 pm
Location: Ever heard of Yoshi's Mansion?

SM63 Level Designer Contest Winner
1st Place in the Runolympics SM63 LDC Event. Congratulations!

Thumbs Up given: 139 times
Thumbs Up received: 102 times

Re: Overhaul of SM63 judging system

Postby NanTheDark » April 30th, 2016, 11:58 am

While thinking about what do "Gameplay" and "Atmosphere" categories actually mean, I thought that maybe it would be best to really think about exactly what is it that people look at when it comes to judging. Not in broad categories, but every single thing that matters of a level. I ain't the best to think of these probably, but I'll try to give my personal opinion as to what is it that people might be considering when judging.

-Difficulty (fair or unfair)
-Level Design (covering platforming and level's overall flow)
-Puzzles (if any)
-Story (if any)
-Graphics (does it look good?)
-Adherence to the Theme
-Whether or not you like that one Alternate Music
-Use of glitches (lol disabling)
-Use of blosses
-Use of FLUDD
-Use of caps
-Lag, probably
-Fun Factor
-"Innovative" concepts
-Cool setpieces
-Tile choice
-Item placement
-Enemy placement
-"Checkpoints", if any. More of a meta thing really.
-Exploration (if it's a big wide level)
-Speed (if it's a sprint from point A to point B )
-Achievements (I can't believe it, but the Achievements I add to my levels sometimes somehow make it into the judgings. XD)
-General Idea

There's probably way more things, feel free to add to it. A bunch of these you can probably place under Gameplay, others under Atmosphere. I dunno. But we should properly define what is it that we're judging as Gameplay or Atmosphere.
Some day I'll fix my sig.

Old Sig Stuff: show
Image

Universal Dueling ArenaInto The ShadowsMy SM63 Levels

Image
Made by FrozenFire/lemontea/whatever. Thanks, FF!

ImageImageImageImageImage
????
Image
User avatar
NanTheDark
As it breaks, it becomes stronger

 
Posts: 2018
Joined: December 23rd, 2010, 6:04 pm
Location: Your screen

Runouw Votes Winner
For winning Best Normal Member in the RV Summer 2017

Thumbs Up given: 191 times
Thumbs Up received: 217 times

Re: Overhaul of SM63 judging system

Postby MessengerOfDreams » April 30th, 2016, 3:14 pm

Remember when the point of this was to "remove bias from the system", not rearrange the bias?
Image
Image

My Most Recent Works: show
I switch my signature a lot. If you wanna see some of my past ones, here you go.
Silent Conversations and a Crow's Final Song!
My latest story, and one of my personal favorites. A girl bound in silence finds the words to say to her prospective girlfriend as they visit her religious father in a dusty town on the edge of Kansas, where the crows' migration south brings forth anchored memories, the path to resolution, and a new start.

Form (25quared)
This might be like nothing you've seen before.
Updated Works! Some of my past best and current stuff: show
Writing Works!
Fanfic: Shut Up and Dance
why do I write so much about dancing you don't dance you've never danced in your entire life
lying little ♥♥♥♥ with your ♥♥♥♥ story ♥♥♥♥ you
also Diddy/Lucina <3

Fanfic: Worth a Thousand Words
Because the world needed a Samus/Dedede story
Fanfic: Ecstatic Silence
Just wanted to write and ♥♥♥♥ like this happens, you'd think I'd know better.
Fanfic: Far From the Edge
It's a dance that's been a long time coming for a brand new man and an unchanging woman, but once one takes the plunge there's no falling back up.
Original: Jealous Ghosts of the Mississippi
The story of Rachel meeting Amber after a lifetime of silence and being shunned
Original: Your Hand in Mine
One of my most personal stories about a dangerous romance. Now to be published in a college lit journal!
-------------------------
Recent LDC Work:
Level Series: Leaves From The Vine (ft Star King)
For the 29th LDC, a theme of grassland taken through the growth of our designing society. Won the 29th LDC!
--
Levels at Large:
Level Topic: Collection of MoD's Levels!
If you've ever wanted to see any level I made worth a damn, go here! From the quiet 14th LDC entrant Finis to the megasmash level series Dark, you can find links here!
"You were always a revolutionary, now there's just less of a chance of you crying in the corner." ~Ridder
User avatar
MessengerOfDreams
Moderator

Error contacting Twitter
 
Posts: 6615
Joined: August 16th, 2009, 11:31 am
Location: When I figure it out, I'll let you know.

Winter
2016 Story Contest

Thumbs Up given: 519 times
Thumbs Up received: 707 times

Re: Overhaul of SM63 judging system

Postby Megar » April 30th, 2016, 3:20 pm

'Removal' of bias is an impossibility because every judge has their preference which they're naturally inclined towards. To assume otherwise would remove the point of having multiple judges as they'd all think the exact same.
User avatar
Megar
Follower of Razputin

Error contacting Twitter
 
Posts: 2482
Joined: December 31st, 2009, 3:08 am
Location: On the Steel Ball Run

Runolympics 2015 Winners

Thumbs Up given: 42 times
Thumbs Up received: 147 times

PreviousNext

Return to Level Designer General