Request for Transparency

[-BY]

This post by -BY was thumbed up by:

Supershroom (April 30th, 2015, 12:07 pm)

[Venexis]

Just said this in IRC, but as a backup, I would love to see this become the norm.

My biggest worry was of not being consistent- I could forgive myself even if nobody else did if I had rated every single level, but that wasn't the case, and I've been known to be a bit more strict than the average in the past. The most stressful part was that one of my scores would fall well outside the "acceptable range" as defined by full judges, and... it kinda happened. I rated MK's level significantly harsher than any other judge, and MoD's/SK's half a point better than anyone else.

I guess when you break it down those gaps aren't that significant compared to levels like CY's, who was affected by nearly 1.5 points, but... it'd definitely be cool to communicate freely with other judges, if not participants (I think there are merits for this too, did I totally miss the point of an abstract level like MKs? Never gonna know for sure unless I talk to the creator), as I at least encountered a significant amount of difficulty trying to not unintentionally ♥♥♥♥ up a score by rating as I typically would as a full judge.

I feel like my scores overall were much higher than normal to attempt to passively compensate for that, by 1-2.5 points depending on the case.

[*Emelia K. Fletcher]

This post by *Emelia K. Fletcher was thumbed up by:

Oranjui (May 25th, 2015, 9:49 am)

[MessengerOfDreams]

https://docs.google.com/spreadsheets/d/ ... sp=sharing

Just some data before we go into accusations:

The placements for each judge show that they each have some levels, around a small handful, that they differed from everyone else on, some moreso than others. Also, nearly everyone had one level that they scored way higher or way lower than average, by anywhere between 2 and 6 points.

However, from what I've found, the average placements almost dead-on reflect the average score, if both were ranked by those averages. If we ranked in a WITBLOAT style, where we ranked from 1 to 18, the only differences would be that Doram and BY switch and some mild shifting in the lower ranks.

I don't think there's negative, biased trends there. That seems to be derived entirely from low personal opinion from some spectators to some judges. As it stands, the patterns are consistent but affect little.

Later, I'll collect data on how scores would have changed under the old Supershroom system if everyone's highest and lowest scores had been removed, to see if it truly can bring equity with 6 judgings.

[Karyete]

Wait, does that graph tell me that MK gave me the highest score of all his judgings? Fo reals?

[Karyete]

This post by *Emelia K. Fletcher was thumbed up by:

Karyete (April 30th, 2015, 11:49 am)

[Karyete]

This post by Karyete was thumbed up by:

~MP3 Amplifier~ (May 1st, 2015, 9:51 am)

[Venexis]

I don't think there's negative, biased trends there. That seems to be derived entirely from low personal opinion from some spectators to some judges. As it stands, the patterns are consistent but affect little.

To be perfectly clear, I agree with this. I don't really think this is necessary, but it sure would be incredibly nice to have, especially as a resource for backup judges.

Like, usually my "average" score is in the 10-12 range as opposed to the commonly seen 13-15 range... that could definitely be interpreted as bias, even if it is unintentional, as those harsher ratings would only apply to the few levels additional judges are needed for. A group communication session would go a long way toward detecting and preventing similar issues that otherwise would not have been evident until after the contest's results post.

[MessengerOfDreams]

Well, judges having higher standards or higher praise doesn't really affect overall standards if that's just how they are unless, like you, they're alternate judges judging only a small amount of levels, or one is simply worried about overall LDC standings comparable through history

Also, I updated my chart significantly. The first is taking the "let's cut off the worst and best judgings" verdict from way back to show it makes nearly zilch difference. The next is just laying out exactly which order each judge took. Some are weird orders, but overall there are just a few special cases in a uniform order.

[Doram]

This post by Harmless was thumbed up by: 2

Doram (May 13th, 2015, 4:30 pm) • Oranjui (May 25th, 2015, 9:44 am)