Suggestion for The Level Portal Rating System

[l.m]

Hello everyone. Now, I've been discussing this with lordpat and some other people and I'd like to show here something of extreme importance to our forums. It doesn't matter if they are made with SM63 or LL. The levels are the base of the forums, and most of the users here are focused into them. Level Designer Contests, WITBLOs, Medals, Tier Lists. Those are all part of our community, which is decreasing and losing some few activity.

But that's not what I came here to talk about. The place where you can find all the levels is the Level Portal. There, users can post their ultimate masterpieces and recieve feedback from another users. And this is where we can find the problem. Ultimately, new users, who doesn't have much experience of level designing, posts bad levels, continues to post them and ends up filling the portal with their creations, and we all really want them to keep getting better and better. And if we want this to happen, we must give feedback.

Now, let's get to the point: What do you really want to say?

My suggestion is: If someone wants to rate and to give feedback to the creator, they must justify. They must explain why and what should you do to make better levels. If you want to rate any level 1 star, you must explain why, and the same happens if you rate 2 or more stars, like this:

Good level. I really liked this and this but you should improve this because it ruins the fun. Also, this isn't very beautiful because this and this. So, I rate 4 stars.

And not like this:

Ew. This level sucks. 1 star.

Not even like this:

OMG AWSOOOOOM 5 STARS NOWWWWWWWWWW

But FrozenFire, why?

1- As I already said back here, we really want new people to get better and better at the level designing scene. And if we want this to happen, we must give feedback. We must show them what is wrong, and what is needed to improve.

2- This system could prevent downraters. All what the downraters do is just click on the 1-star icon rating and then fly away like nothing happened. Of course, mods could simply give them a warning or maybe even a ban for them, but this would decrease a little bit their job in this area.

3- Please, look back at your past. I'm pretty sure you didn't make a great level right at the start. Probably, someone gave you constructive criticism. And you accepted it, you grown, and now you're famous. Don't we want this to happen with all the new users too?

Aw, come on, FrozenFire, I don't want to write a thousand-paragraph review everytime!

You don't have to. Really. You don't need to waste a day on giving constructive criticism to everyone. You just need to write some lines, like this: "Good job, I liked this and this in your level, but this didn't work out in your level, and I don't think this was beautiful, so I'll give it x stars."

But your suggestion would decrease a lot of the feedback given!

Hey! You don't have to help our community grows, but if you do and give feedback, you'll be rewarded. Medals should be given for people who really contributes with giving feedback for new people and helping the community grow, replacing the old ones which are based on the star rating system. So, it won't be a total loss of time.

But there's still one problem: There's a lot of new users who don't even care about feedback!

Yes. Of course this is a problem.

You know, everyone have a chance of hearing and trying to to better using the constructive criticism. But what if the user doesn't cares and continues to post crappy levels on the portal?

That's where my rating system suggestion enters. If any user gets past the approval system (where their levels must be approved) and then posts 3 levels that are poor and with no effort at all according to the ratings given, they should go back to the approval system. Meaning that even if they have their levels approved and have past the approval system, they must continue to post good levels, and if the latest three of them are bad, they should go back to the approval system and their new levels must be approved by the level moderator. Of course, after three of them are approved and OK, they should be able to post levels normally.

But, FrozenFire, I'm scared!

No, you don't need to be scared. Everything will be OK. Trust me.

That's it. That's my suggestion for the rating system. I'd like to know if this is a really good idea, or if there are flaws with this system and there are things here that should be improved.

[~Yuri]

Well, while this is a good idea, I don't think it will work really hot.

Most people who rate levels on the portal won't be bothered to write a comment everytime they rate a level, resulting in the amount of rating decrease a lot. Seriously, rating must be something as easy as possible, so more people (especially new members) are incouraged rate. New members on the portal most of the time don't really care about the forums, so giving him a medal for reviewing levels is pointless, and wouldn't really encourage them anyway.

I've said countless times though, how much I hate this approval system though, because of how it allows any kind of level in after three approved levels. I think the approvation should be based on the ratings of any level the user has approved + the ammount of levels it has disapproved, to avoid more bad levels into the portal. But unfortunately, without Suyo to code stuff like this for us, this would be impossible.

Maybe, your system could work for way too high/too low ratings, but I think it wouldn't work really well for the whole rating system.

EDIT: Moved to the Level Portal General forum

[MessengerOfDreams]

I do love it in theory but I don't think it will change anything other than the only ratings will be from the people who'd review anyways.

[TheExcavator]

The reason I asked someone to start this discussion here off of IRC is because I noticed a small thing that I consider to possibly be a problem. This thing is the ease at which one can play a level, feel slightly motivated to give feedback, and just rate the level and feel like they did something when in all actuality they have done little more than click a little icon on the thread of the level they felt motivated to do something towards.

Why is a rating worth very little? It's pretty simple really, it's just a number. The designer can look at that number and determine very little. If the number is low they can probably work out they did something very very wrong but not precisely what that something is. At this point, assuming no one bothered to post actual feedback the designer can only guess what that something is and then trial and error occurs, and the designer either finally figures out what that mistake was or gives up. The reverse is that a newer designer makes something and that level recieves relatively high scores. Okay. What did the people like about it? The number does not answer that question. Once again, they are left to guess.

That is what I think is a problem. If I had it my way I'd abolish ratings entirely but that runs into it's own problems I won't get into. If nothing else comes out of this I would like to see only this: Actively encentivize and reward giving actual words; useful feedback and advice that the designer could possibly use. Because as of right now as far as I am the only rewarded method of giving "feedback" is the "give X ratings" medals, and if you were paying attention one paragraph ago you would recall that ratings are near useless.

tl;dr I wanted to talk about a problem. This problem is that the most convinient method of communication with designers old and new is the worst method of communication. Please please please actively encentivize giving real feedback.

Offtopic: Dont expect me to post all that much I am using convoluted methods to post from my phone to ensure I dont have to type every word I wrote here on my phone. This offtopic message was written on my phpne and it hurt.

[Raiyuuni]

<Raiyuuni>: FF was rambling on yesterday about feedback quality while only putting community reputation and square medals as reward
<lordpat>: backity back
<Raiyuuni>: lordpat, hear this out
<Venexis>: I dunno what more could be done, beside approving users to rate levels
<Venexis>: Which frankly seems like it'd become not worth the trouble to all but the most dedicated members of the community
<Raiyuuni>: I'd suggest that level submission and grading regulations needed some overhauling
<lordpat>: overhauling how?
<Raiyuuni>: well, first off, we'd need to add another step to level submission filtering
<lordpat>: hmmm
<Raiyuuni>: I can't find the exact words for this, but grading maps would return a set amount of base points depending on who has created it, how difficult is it (this grading would also be a responsibility of level mods) and how many people have already graded it
<Raiyuuni>: and you'd need a set amount of points to be able to submit new creations for approval
<Raiyuuni>: but, here's the thing. This activity multiplier, along with user levels, would make this system more lenient on newcomers, while being more demanding on active/experienced users.
<Venexis>: Something like, User A drops a review for Level B, Level Mod C comes by and checks that out, then assigns User A a certain number of points that may unlock additional abilities?
<Raiyuuni>: comment approval might be a bit overkill at the beginning
<Raiyuuni>: and points would be automatically granted, which is the main obstacle of such a system which I'll elaborate on later.
<Venexis>: Lawl yeah, but just as a general template of the sort of interactions that would be required
<Raiyuuni>: instead of comment moderating, we'd have to create a template for rating
<Raiyuuni>: in which the necessary criteria would be written individually, even if on very short comments
<Raiyuuni>: I don't know either how safe would it be to grant access to the "Other" criteria to the very, very beginners
<Raiyuuni>: so they'd just drop a 3/5 by default until a set landmark
<Venexis>: Makes sense.
<Raiyuuni>: Also, "player levels" would be drained constantly, and even more so the longer the break is
<Venexis>: As in, the overall rating for that level would slowly go down, or?
<Raiyuuni>: but they'd grant more points on already existing medals, which would return even more points
<Raiyuuni>: oops
<Raiyuuni>: Along with the "activity rank" necessary for submitting comments, doing so would also increase "player ranks"
<Raiyuuni>: and the former can also be done by clearing maps which received high difficulty ranks from Level Mods
<Raiyuuni>: This system would also be used to solve a second issue: judge selection.
<Venexis>: Clever.
<Raiyuuni>: Active users with good feedback regarding their feedback would be granted a "level advisor" rank, intermediate between normal members and staff members.
<Raiyuuni>: That allows them to approve levels on Level Mods' stead, and apply to be judges for LDC's.
<Raiyuuni>: Normal members can also apply, but they would have to be accepted by any staff members.
<Raiyuuni>: However, there would be no restrictions regarding judge selection for duels.
<Raiyuuni>: And here is the most tricky part of all of this:
<Raiyuuni>: The "activity rank", "player rank", default awards, event awards (by taking part or judging on events) and weighted level ranking average would return a global value
<Raiyuuni>: from which a global rank board, parallel to the LD Tiers, would constitute of
<Raiyuuni>: Newcomers would be drawn in further to more serious competition through having their very nicknames being featured on that table
<Raiyuuni>: except all of this would have to be performed in order not to lock out members on both age extremes:
<Raiyuuni>: the youngest, by being less knowledgeable on level designing or, more frequently, on achieving proficiency on a foreign language in order to take part of all this
<lordpat>: .define test
<B-O-T>: test — noun: 1. A cupel or cupelling hearth in which precious metals are melted for trial and refinement, 2. A challenge, trial — verb: 1. To refine (gold, silver, etc.) in a test or cupel; to subject to cupellation, 2. To challenge
<lordpat>: ty
<Raiyuuni>: and the eldest, which have greater responsibilities on their jobs/studies, or simply cannot maintain as much interest on the system as they did before, and thus cannot spend as much time adding to the LD scenario.
<Raiyuuni>: analogously, we'd have to be careful not to turn all of this into an MMO in disguise so that there would be people grinding reviews all over the place, and further increasing the gap between veteran and rookie userbases.
<aznchau4ever>: ♥♥♥♥♥
<Raiyuuni>: That's about it. :C
<Venexis>: Just to recap
<Venexis>: Complete overhaul of the level system- introduction of different scores that serve as indicators of a user's involvement with the Level Designer, which can be increased by participating in certain tasks. As a user gains more points, they gain new abilities and new ways to get more points. The points can then be used to determine ranking systems for things like LDC judges, or just to serve as a motivator?
<Raiyuuni>: both
<aznchau4ever>: Hyperdimension Neptunia pushed back two more days
<aznchau4ever>: cool
<Venexis>: Sorry, that was all meant to be one summary lawl
<Raiyuuni>: The global statboard would be the ultimate goal, and reviews are only another way to move up in the table.
<Raiyuuni>: And we still can't turn this into an MMO by granting ridiculous offsets to old players.
<Venexis>: I'm gonna need to head out right away here, but it sounds like a super useful way to sort out a lot of the current... unpleasantness... in the LD scene right now, among other things. Problems would be, as you mentioned, implementing it so that it was balanced against members of all experience levels
<Venexis>: That and I'm not sure how easily it could be implemented, if at all
<Raiyuuni>: That's the big deal
<Venexis>: Usually Suyo would be the code guy for all the new systems
<Raiyuuni>: we need to enslave more Yoshis in order to even consider applying something along those lines
<Venexis>: Lawl
<Venexis>: I was thinking, maybe a consideration for the new Last Legacy site?
<Raiyuuni>: That's a place where I haven't been in a while. :C
<Venexis>: But I don't really have time to make a post or anything right now, I'm gonna be leaving in about ten minutes. I'll definitely pass it along then though, unless someone beats me
<Venexis>: .in 2h Ven check two hours ago thanks
<B-O-T>: Venexis: Okay, will remind at 01/28/15 - MST 13:40:40
<Raiyuuni>: can I just copy-paste all of this
<Venexis>: Lawl if you want.
<aznchau4ever>: Hello
<aznchau4ever>: My name is
<aznchau4ever>: austin chau

[l.m]

That's a clever idea you had. We also should care about users who just gives unuseful feedback just for the points and the rank.

Also, like Yurimaster said, the new users are there just for the Level Portal, and to post levels, and sometimes they ignore any feedback. Then, your feedback system would just give points to the User who gave feedback, but it wouldn't benefit the creator of the level.