runouw.com

Something that I've been disccussing with Yurimaster is an issue that seems to be becoming more and more noticeable with time. It's something that, if left unchecked, can actually start to affect the integrity of all future LDCs, SM63 or Last Legacy.

Star Power is a term that was kinda popularized by the Guitar Hero games. In general use, it kind of got the meaning of using your "popularity" or "stardom" to get through something.

What am I refering to?

There's a situation that has been repeating in the latest LDCs, where the ones at the top always seem to be... the same. There's 4 or 5 names that often keep showing up. Like, MoD, Volkove, MP3, Star king... off the top of my head. And take a look at the Tier List. Look at the first four places. That's just a freaking coincidence, I wasn't looking at the Tier List when I wrote their names.

Sure, there's most likely a reason they're there. They do make excellent levels, I'm not gonna argue with that. However... no matter what, whoever's on top, will always stay on top, and new LDers rarely have a chance.

This post from Venexis sums it up quite nicely.

Venexis wrote:Judgings. But yeah, I have kindof an issue with the generally accepted review method too- An average level should be exactly 10/20 points. This, to me, would describe perfectly a playable level, but with no special effort put in whatsoever. Nothing wrong with it, but nothing to merit a higher score either. 18/20 or so would become the elusive "master designer" score (with only 3/5 in Other, as everything else would usually be covered- anything higher would be virtually godlike). That's radically different from how the current system is, with an average level being about 12/20 and "top-tier" work being a perfect 20/20. Because of the way it's set up (The other category not being on a sliding scale, for instance) a score should literally never be higher than 18.5... and yet it seems to happen regularly.

I guess what I'm trying to say is that things such as the above, while it only results in a difference of a few points, introduces a subconscious bias in judges (looking back I think this is particularly evident in my reviews). Instead of fixing a midpoint and rating a level in terms of how far it falls away (either above or below the average), we seem to have fixed only the perfect scores, and rate levels in terms how much worse they are. You'll notice how a new competitor in LDCs very rarely steals the show, but the universally accepted great designers tend to remain in the top places? They are the manifestation of that elusive 10/10, and should someone actually do better than them in any aspect, there is no higher score available.

...

tl;dr: don't subconsciously pick a winner before even starting to write your reviews, judge objectively and relative to a neutral midpoint. It's going to affect everyone, and that's why it's critically important in order to praise the designers that deserve it and not necessarily the ones who have been praised in the past.

Judges do seem biased towards the Designers who already seem to be awesome. The Tier List is actually kind of a good testament to that, just look at it, and you'll see that the higher one is in the Tier List, the most likely they are to win. A Bronze Tier just doesn't stand a chance against the people above.

I'm not saying that this bias might be on purpose, it might be unintentional, just like Venexis says. In the end this might just be a problem about the way Judges approach their job.

Maybe we should start thinking about better ways to approach the reviewing process. Define a standard or something.

Comments, suggestions, opinions, flames, all down below.

The only oddity of the past 5 LDCs (mini-LDC's excluded) is Sympathy For The Devil IMO. I'll never like this, there where other levels in this LDC I've liked much more, and I'm aghast at judges writing at the fourth level "+1 for no lag" or something like this while I get less than 10 fps on my computer though it's good. But everything else was pretty fair. -BY, Ayrayen, JSlayerXero, KABOOM and Yurimaster have won their first trophies, so I don't think newbies or returners don't have any chances. Let's see how my debut turns out.

For the judging system itself, I think it's well-proven, it shouldn't be changed. Any judging styles in the topic of the same name can be comprehended.

...sorry, are we back in Kindergarten where everyone gets a turn to win? I'm always thrilled when a new designer or new winner wins but they have to earn it. I don't think people should be penalized for doing well or I may as well not design out of fear of being guilt tripped. We had the anon vault going on last LDC which was inexplicably cancelled and Amp still won. Maybe we're actually good designers And we aren't rigging they system. Maybe people bring in negative bias against people they don't like or don't want to see win- I can expect judges now to be extra harsh on me if they don't want a new winner now.

Also thanks for specifically calling my level out Shroom. Can always count on you to bag on others.

MessengerOfDreams wrote:Also thanks for specifically calling my level out Shroom. Can always count on you to bag on others.

I would also not like it if it was from any other designer, it has nothing to do with you especially and I know it for ages. I judge everyone's levels equally and I promise I'll judge everyone fairy this LDC.

In my honest opinion, people have some bias toward high ranking people in the tier list indeed. Ok, you guys even deserve it at some point, but seriously, I've seen way too much bias on some judgings recently. I won't say I'm excluded of that, because I admit, I had some bias toward some people in my 25quared judgings. I don't mean to offend anyone, as Ven said, it's something we unconsciously do, with no intention of working toward some, or against some. it's just that the "Oh, I know this person, I'm a big fan of him, so I'll give him a bigger score." or "Meh, this is a portal designer, what chance does he have against the veterans?" affects some people.

Really, how many times a veteran placed without even wanting to? Or some people who got in the tier list because of one good level, while some portal designers never got there even though they have some amazing levels?

Personally, the system in the 26th LDC would have worked, but the problem is not with making the entrants anonymous, people could easily recognize by style. We should pick judges that doesn't know any of the contestants or the styles at all, someone who is not from this site, but experienced and skilled enough with level designing, to tell what is good and what is bad, without having bias for any high tier level designer.

I feel like people are ignoring that as Shroom said, between the 20th and 26th LDCs, despite the infestation of Amp wins, Buffooner, BY, Ayrayen, Nwolf, and Yuri have all won their first gold. In one LDC both Amp and SK bombed in the lower ribbon section below MegaR, BY, and Kaboom, three relative rookies. And hell, maybe some of those were influenced by personal bias. Ayrayen had one of the best level series ever, but he also had the story about coming back to the site after a career of being a mid tier designer in the early days, publicly fighting depression, and then blowing everyone away on an LDC. Maybe people thought BY finally deserved a medal after being a portal legend.

There's no question that Amp, Volkove, SK, and I have done reasonably well. But in the past 7 major LDCs, we've taken not that many of the medals. The 20th had SuperMIC and Buff, the 22nd had BY and Kaboom, the 23rd had JSlay, and Nan, 24th had Ayrayen and Killswitch, 25th had Nwolf and Doram, and the 26th had Yuri. It hasn't been a monopoly.

uh

all I did when I judge was just judge what was in front of me ._.

dont hrt me pls

I think all of this is misunderstood...if I'm completely honest, I'm actually slightly stricter on higher tier list designers. That for me is a bias I can't help, because I take their high position into account so when they make the mistakes the same as others I may end up taking off more points on their levels, and it's not something I intend to do, just something I do subconsciously.

Thing is this topic is assuming that everyone is biased towards higher tier list designers. But before late 2011 there wasn't even a tier list at all, and the same people still won awards for their efforts. Did Murph place in those first LDCs, because people secretly wanted him to? Did Four win twice in a row because people liked him more than the others? A perfect example is Volkove, I've actually won more times than him (not bragging lol) but he's still top of the tier list. The first time I personally ever won, I was like 20th on the list and I really worked hard to get up to joint 3rd where I am now.

I'm not saying the rest of you don't work hard enough because I see potential in all of you, but I'm partly in agreement with MoD that you really need to earn a win, and newbies have won before so it is all possible.

It's just if this kind of thing continues, we may start to have this thing where we're all more lenient and generous with scores on newer designers just so they can have a 1st place trophy. In some ways, you want a harsh judge really, because you can then learn from the mistakes they pointed out. That's mostly the reason why I achieved so much.

I hope this post isn't taken the wrong way at all, I've tried to be balanced with my arguments. It's just the way I see it, like every other competition out there, you have to earn your win. And personally I only ever feel like I deserved to win once (maybe twice). I understand that it's hard because believe me there was a time where I was desperate to win an LDC, 16th LDC entrants and before will remember that. But long term I used this frustration as a way to motivate myself, instead of just letting it cripple me like I could have done. And also, while I don't really mind right now, having my name mentioned as one of the returning 'legends' that constantly places makes me feel a bit guilty for winning so many times.

Supershroom wrote:... and I'm aghast at judges writing at the fourth level "+1 for no lag" or something like this while I get less than 10 fps on my computer though it's good.

The only cause for this is that I had another level opened at the same time. Strikethrough'd, it doesn't lag by itself. Sorry, bro.