Request for Transparency
Moderator: Level Moderator
Re: Request for Transparency
Thumbs up x2 
by nin10mode » May 26th, 2015, 9:47 am
MessengerOfDreams wrote:And behavior like this condescension, anger, demand, entitlement, and strongarming is likely why I'll never judge again. Any good idea that may be there is lost when you know the reactions will not be acceptance and understanding, but flat out ♥♥♥♥♥♥♥ like this.
Guess what? It's a two way street. Are you not going to admit you've been an ♥♥♥♥♥♥♥ as well if you're going to call this one instance of directed frustration the musings of an ♥♥♥♥♥♥♥ (I'm fairly certain this is the first time I've actually called anyone a name in a year)?
I'm really tired of dodging the bullet so I'll just go ahead.
You and Shroom have been antagonizing each other for almost a year, people are taking sides and only a select few have tried meeting in the middle. These are not opinions and controlled debates, these are passive aggressive attacks and stubborn attitudes towards potentially helpful changes based solely on the presentation of the idea.
"Shroom's name is attached to this idea. Therefore it must be a bad idea."
That said (the heat's pretty much cleared between us already guys, don't worry), here's a process that might work (I don't believe a process was given for the 25quared), grossly simplified:
1. Normal judging phase
2. Everyone PMs each other their reviews
3. Everyone takes notes on what they have problems with
4. Discussion phase
- -Everyone presents their list of potential problems
-repeats are tallied
-Discussion: least mentioned -> most mentioned
-Original judge of problem justifies/reevaluates
-Repeat
-
nin10mode - Immune to Death
Error contacting Twitter 
Error contacting last.fm - Posts: 3087
- Joined: October 3rd, 2009, 6:11 am
- Location: Not here
Credit To Team - Thumbs Up given: 102 times
- Thumbs Up received: 221 times
Re: Request for Transparency
Thumbs up x3
by Doram » May 27th, 2015, 5:45 pm
Doram readies his flamewar-extinguisher...
Whoah. All right. Listen. This IS starting to get out of control. I have literally let this post sit all day as I have worked on it, in order to remove all the harshness of my own reactions, because they will not help this situation any more than the bruised feelings of everyone else.
BY: I see what you're talking about, but we were already moving in this direction anyways. That's why we all post the judgings after the scores are released, so people have the justifications for the scores in words. There's transparency there already, for the most part. I can see us making it so that the judges HAVE to post SOMETHING as a text-based judging, but that is probably the most that this needs. Yes, a bit of discussion before the final display might be useful, but in the long run, part of the point of the judging process is that people are expressing how they feel about a level, and sometimes, that is just going to bring some randomness and differences to the mix. Further explanation follows in response to other people's concerns...
Shroom: I know it has been said that we are perhaps resistant to new ideas. I say again, that we can review some of this, but I do not guarantee change because the simple fact of life is that newer does not necessarily mean better. Furthermore, I do not appreciate anyone saying that we are not being fair about things with the way that we do them, because we have, in fact, worked very hard to be fair, and are doing the best we can. It is insulting to even hint that you think anyone is being unfair about this stuff, and that is what MoD is responding so strongly to, especially the thought that he has to change the way that he views the game, because someone else thinks differently. It is perhaps amplified by other events in his life, but I wholeheartedly agree with the sentiment, that he has fought too hard and for too long to have someone tell him how he should live his life. And finally (as I do believe I have said before), I have studied Statistics, and Human Psychology, and I can concretely say that a number of the suggestions I have seen you make are actually counterproductive to your stated goal. This seemingly broken logic, especially when so forcefully delivered is one of the reasons I have gotten quite mad at you, and let me say, it is VERY hard to get me mad.
Ven: Good gods, man. Everybody's up in arms over this. I was with you right up until that last paragraph. Perhaps they shouldn't be judges? That's a bit too harsh.
Nin: I am also dismayed at how badly this has gotten to you, sir. You and Ven are two of the most cool heads around here, and this has actually gotten both of you completely on edge. The situation needs neither condescending laughter nor real anger. I agree that this is being blown out of proportion, and some things do not seem to make sense, but the truth of this is much simpler.
MoD: Stand down, sir. Please. Asking for your justification of a score is not the same as telling you that your scores must conform to anything. That is all that the reasonable people are asking for, and the unreasonable are about to hear otherwise.
Because that is the point at which my support of all this breaks down. I do NOT believe that the scores need to be massaged into conformity. If the person has a justification for it, there is no reason that Figure 2 is not acceptable. Disagreements happen all the time around here, and that is no reason to tell a person that they have done something wrong by scoring the way they did, because they haven't done anything wrong. They looked at it, and it was a 10/20 for them. That's it. It isn't good, bad, or indifferent. It's just what they thought about that level. As has been mentioned many times in this conversation, people are different, and this means that their view of the level will be different, and that's a GOOD THING. As has been mentioned by multiple people, as long as there is a justification, there is no reason to modify ANY score. The staff picks judges based on their experience in the field, and their maturity and ability to dispassionately rate something. That's how they get there. They are not going to randomly campaign for someone's failure, and they are not going to do anything other than tell you what they see. That's the job, and it's been done just fine this whole time. Outliers happen, and yes, it is sometimes a matter of several people not letting the flaws overwhelm their enjoyment of the level, and one person saying "No. That broke it for me." Ok. the last guy gave it a 10/20 because he didn't have fun.
didn't have fun.
didn't have fun.
didn't have fun.
...which brings me to my last point.
THIS IS A GAME. CALM DOWN.
Whoah. All right. Listen. This IS starting to get out of control. I have literally let this post sit all day as I have worked on it, in order to remove all the harshness of my own reactions, because they will not help this situation any more than the bruised feelings of everyone else.
BY: I see what you're talking about, but we were already moving in this direction anyways. That's why we all post the judgings after the scores are released, so people have the justifications for the scores in words. There's transparency there already, for the most part. I can see us making it so that the judges HAVE to post SOMETHING as a text-based judging, but that is probably the most that this needs. Yes, a bit of discussion before the final display might be useful, but in the long run, part of the point of the judging process is that people are expressing how they feel about a level, and sometimes, that is just going to bring some randomness and differences to the mix. Further explanation follows in response to other people's concerns...
Shroom: I know it has been said that we are perhaps resistant to new ideas. I say again, that we can review some of this, but I do not guarantee change because the simple fact of life is that newer does not necessarily mean better. Furthermore, I do not appreciate anyone saying that we are not being fair about things with the way that we do them, because we have, in fact, worked very hard to be fair, and are doing the best we can. It is insulting to even hint that you think anyone is being unfair about this stuff, and that is what MoD is responding so strongly to, especially the thought that he has to change the way that he views the game, because someone else thinks differently. It is perhaps amplified by other events in his life, but I wholeheartedly agree with the sentiment, that he has fought too hard and for too long to have someone tell him how he should live his life. And finally (as I do believe I have said before), I have studied Statistics, and Human Psychology, and I can concretely say that a number of the suggestions I have seen you make are actually counterproductive to your stated goal. This seemingly broken logic, especially when so forcefully delivered is one of the reasons I have gotten quite mad at you, and let me say, it is VERY hard to get me mad.
Ven: Good gods, man. Everybody's up in arms over this. I was with you right up until that last paragraph. Perhaps they shouldn't be judges? That's a bit too harsh.
Nin: I am also dismayed at how badly this has gotten to you, sir. You and Ven are two of the most cool heads around here, and this has actually gotten both of you completely on edge. The situation needs neither condescending laughter nor real anger. I agree that this is being blown out of proportion, and some things do not seem to make sense, but the truth of this is much simpler.
MoD: Stand down, sir. Please. Asking for your justification of a score is not the same as telling you that your scores must conform to anything. That is all that the reasonable people are asking for, and the unreasonable are about to hear otherwise.
Because that is the point at which my support of all this breaks down. I do NOT believe that the scores need to be massaged into conformity. If the person has a justification for it, there is no reason that Figure 2 is not acceptable. Disagreements happen all the time around here, and that is no reason to tell a person that they have done something wrong by scoring the way they did, because they haven't done anything wrong. They looked at it, and it was a 10/20 for them. That's it. It isn't good, bad, or indifferent. It's just what they thought about that level. As has been mentioned many times in this conversation, people are different, and this means that their view of the level will be different, and that's a GOOD THING. As has been mentioned by multiple people, as long as there is a justification, there is no reason to modify ANY score. The staff picks judges based on their experience in the field, and their maturity and ability to dispassionately rate something. That's how they get there. They are not going to randomly campaign for someone's failure, and they are not going to do anything other than tell you what they see. That's the job, and it's been done just fine this whole time. Outliers happen, and yes, it is sometimes a matter of several people not letting the flaws overwhelm their enjoyment of the level, and one person saying "No. That broke it for me." Ok. the last guy gave it a 10/20 because he didn't have fun.
didn't have fun.
didn't have fun.
didn't have fun.
...which brings me to my last point.
THIS IS A GAME. CALM DOWN.
Martin Luther King Jr. wrote:Man must evolve, for all human conflict, a method which rejects revenge, aggression, and retaliation.
The foundation of such a method is love.
-
Doram - Global Moderator
- Posts: 1524
- Joined: February 22nd, 2010, 7:37 pm
- Location: Wherever I'm needed.
Cookie - Thumbs Up given: 153 times
- Thumbs Up received: 471 times
Re: Request for Transparency
Thumbs up x1
by Harmless » May 27th, 2015, 6:24 pm
Yeah, I'm sorry, I can't hide it anymore.
I purposely delayed my judgings for the most recent 25quared level contest because of this blowout, and with all the arguments going on daily. With all of this sudden controversy and people yelling at eachother (myself included on both ends), I didn't feel comfortable sharing my judgings at all.
I'm sorry. I'll get them out soon.
I purposely delayed my judgings for the most recent 25quared level contest because of this blowout, and with all the arguments going on daily. With all of this sudden controversy and people yelling at eachother (myself included on both ends), I didn't feel comfortable sharing my judgings at all.
I'm sorry. I'll get them out soon.
~ Tesla Bromonovich
-
Harmless - Is it lunch time yet?
- Posts: 2793
- Joined: June 25th, 2011, 11:53 am
- Location: Mother Russia!
Runouw Votes Winner - Thumbs Up given: 271 times
- Thumbs Up received: 240 times
Re: Request for Transparency
by ~Yuri » May 27th, 2015, 6:53 pm
I'm sorry, but no, such a discrepancy between scores is not really fine. If all the other judges give the level scores ranging between 15-18, and you give it a 9, then what did happen here so the other judges did have fun with the level, liked it, and you didn't? Just saying "I didn't have fun", isn't enough. You can even say the reasons you didn't have fun, but what even happened that the other judges had fun and you didn't? That's what the discussion is for, to settle things like that. If you have a valid reason for your scores, then you should be able to argue about them with no problem, even though there's a difference.
And remember, an official contest is not about which level is your favorite, it's about which level is objectively better. It's really easy to tell if a level is better than another regardless of your personal favorite. The one thing we don't want here is unfairness. It's ok to give or take extra points for a level for your personal fun, as long as your score still counts the concepts, and errors. If you simply praise a level even though there clearly are problems with it, then it's not fine. If you bash a level, but points outs all its good points, innovations, yet doesn't give the right, according score, then it's not right.
This solution is the best we could have for this problem, honestly. Will it take time? Yes. It will. But from now on, it's your job as a judge to participate on that discussion, just like it's your job to judge levels.
And remember, an official contest is not about which level is your favorite, it's about which level is objectively better. It's really easy to tell if a level is better than another regardless of your personal favorite. The one thing we don't want here is unfairness. It's ok to give or take extra points for a level for your personal fun, as long as your score still counts the concepts, and errors. If you simply praise a level even though there clearly are problems with it, then it's not fine. If you bash a level, but points outs all its good points, innovations, yet doesn't give the right, according score, then it's not right.
This solution is the best we could have for this problem, honestly. Will it take time? Yes. It will. But from now on, it's your job as a judge to participate on that discussion, just like it's your job to judge levels.
credits to Nan for this amazing gif ^-^
other sigs: show
-
~Yuri - Boo
- Posts: 677
- Joined: December 5th, 2012, 3:09 pm
- Location: lost, can you please give me some directions?
Cookie - Thumbs Up given: 46 times
- Thumbs Up received: 85 times
Re: Request for Transparency
by MessengerOfDreams » May 27th, 2015, 6:56 pm
Wait... So are we with Doram that being able to defend a discrepancy is okay or with Yuri that its absolutely unacceptable?
Also forcing judges to define objective like we're market testers forced to meet a criteria instead of people trying to find the most enjoyable level is literally what I'm talking about so if you can stop treating me like a tinfoil freak.
Also forcing judges to define objective like we're market testers forced to meet a criteria instead of people trying to find the most enjoyable level is literally what I'm talking about so if you can stop treating me like a tinfoil freak.
-
MessengerOfDreams - Moderator
Error contacting Twitter - Posts: 6615
- Joined: August 16th, 2009, 11:31 am
- Location: When I figure it out, I'll let you know.
Winter - Thumbs Up given: 519 times
- Thumbs Up received: 707 times
Re: Request for Transparency
by NanTheDark » May 27th, 2015, 7:02 pm
IT'S NOT ALL BLACK AND WHITE IT'S GRAY IT ALL DEPENDS ON THE SHADE JUST SEARCH AND LEARN HOW TO DISTINGUISH
Random post to remind people to stop taking sides to argue against each other and instead try to work together to figure out a solution.
Random post to remind people to stop taking sides to argue against each other and instead try to work together to figure out a solution.
-
NanTheDark - As it breaks, it becomes stronger
- Posts: 2018
- Joined: December 23rd, 2010, 6:04 pm
- Location: Your screen
Runouw Votes Winner - Thumbs Up given: 191 times
- Thumbs Up received: 217 times
Re: Request for Transparency
by MessengerOfDreams » May 27th, 2015, 7:06 pm
I thought we were good until a level mod said that we needed to be required to judge objectively and outliers weren't okay. This both proves what I was afraid this would become and spells out that apparently judgings need to all be one criteria. That's not something I am settling with.
-
MessengerOfDreams - Moderator
Error contacting Twitter - Posts: 6615
- Joined: August 16th, 2009, 11:31 am
- Location: When I figure it out, I'll let you know.
Winter - Thumbs Up given: 519 times
- Thumbs Up received: 707 times
Re: Request for Transparency
by ~Yuri » May 27th, 2015, 7:11 pm
Did I ever say differences were absolutely unacceptable? In fact, if you read through my post, I said that the discussion is there for the people who had bigger discrepancies to at least explain why such scores were given. If you can give a reasonable explanation about your score, and take all elements in consideration, not just your frustration/enjoyment, then fine. If you can't, then you'll just have to reconsider your score. Simple.
If you really dislike the discussion process that much, then fine, just don't judge. Again, nobody is ever forcing you to do anything here. And it's not "one level mod", it's the majority who decided we need change. Just look at the poll.
If you really dislike the discussion process that much, then fine, just don't judge. Again, nobody is ever forcing you to do anything here. And it's not "one level mod", it's the majority who decided we need change. Just look at the poll.
credits to Nan for this amazing gif ^-^
other sigs: show
-
~Yuri - Boo
- Posts: 677
- Joined: December 5th, 2012, 3:09 pm
- Location: lost, can you please give me some directions?
Cookie - Thumbs Up given: 46 times
- Thumbs Up received: 85 times
Re: Request for Transparency
by Harmless » May 27th, 2015, 7:33 pm
I actually changed my mind because the change was implemented a bit wrongly, I feel like if there is any bias or fallacies in the judgings after they're shown THEN it should be talked over and rectified if necessary.
Which is pretty much the system we have now.
Though I do agree that if you can justify your score and defend it, there should be no worries.
Which is pretty much the system we have now.
Though I do agree that if you can justify your score and defend it, there should be no worries.
~ Tesla Bromonovich
-
Harmless - Is it lunch time yet?
- Posts: 2793
- Joined: June 25th, 2011, 11:53 am
- Location: Mother Russia!
Runouw Votes Winner - Thumbs Up given: 271 times
- Thumbs Up received: 240 times
Re: Request for Transparency
by MessengerOfDreams » May 27th, 2015, 7:38 pm
First off, the poll was almost dead even so stop trying to act like everyone is on board. Second, you're advocating for a system that encourages "objective" judging which sounds a lot like you're not asking for the many different perspectives judges have, but for one uniform criteria. That's what I'm getting into. Third, falling on "if you don't like it get out" is the sign of a bad leader. And I expect more from you of all people.
-
MessengerOfDreams - Moderator
Error contacting Twitter - Posts: 6615
- Joined: August 16th, 2009, 11:31 am
- Location: When I figure it out, I'll let you know.
Winter - Thumbs Up given: 519 times
- Thumbs Up received: 707 times