A New Moderation System?
Virgo47 popped into the Mapraider IRC chat the other night and proposed an interesting solution to Mapraider's somewhat bumpy moderation system. As I've mentioned before, there are instances where ratings are applied to a map's score when it doesn't get moderated enough times or at all. This is problematic because In the current system, once a rating is moderated fair or unfair, it's permanent (unless an administrator puts it back into the moderation queue).
His solution was quite elegant and has been dubbed the "OMS" — Open Moderation System. The principle is this:
Ratings are never permanently moderated. Instead, their moderation can change over time, fluctuating as more people rate each map and moderate the ratings.
In other words, the moderation of each ratings will change as more people contribute their thoughts, rather than being fixed once a minimum number of moderations have been reached. The map scores will also be more dynamic and better representations of a group's ratings.
I really like this approach as it gets rid of the issue of weird ratings indefinitely applied to a map score and also allows ratings to be instantly applied to the maps rather than waiting for a specific number of moderations. There are, however a few sticking points to be flushed out:
-
The number of moderations in the database will increase exponentially
with ratings, using up valuable database storage.
To get around this, my initial thought was to stop moderations for a rating once the difference between fair/unfair moderation count reaches a specific value. This value would either be a ratio or a percentage of the total number of ratings for the map. For example, 2:1 or 40%, which would make the gap larger with a higher number of ratings. I'd also probably specify a minimum number regardless.
I'm not sure if this makes the new system too much like the old, but in any case, I really feel the need to have some way to avoid using up database storage for moderations. -
An issue arises when there is a tie between moderations — fair or
unfair?
In this case, it seems to make sense that a tie is considered fair. -
Obviously unfair ratings will always start out as fair
I'm not sure if this is really an issue. On one hand, this means that perfect 10 scores on crappy maps may happen more often. But on the other, because they no longer "time out" and become permanently fair, they can be moderated out more easily and the trade off may make sense.