Discussion:I think it's time to change the rating system

I think it's time to change the rating system

The page rating system simply does not work, for the following reasons:

• Many users don't seem to understand how it works. This causes the following issues:
  • Neglecting to remove the REMOVE THIS LINE line. Frankly, if someone can't even bother to proofread their own edit, how can we be sure it's valid?
  • Misunderstanding the purpose of the <<<brackets>>>. I think users seem to think that they perform some sort of functionality, which leads me to...
  • Neglecting to update the rating on the main page. Users, especially new users, probably don't understand that this is a manual process. Add to that the fact that you have to get out a calculator, and that the template is not very clear to someone who's never worked with templates before, and you basically end up with the all-but-guarantee that any rating on the site is wrong.
• Ratings are too static
  • If an IP rates a page, citing a criticism, and then someone fixes that criticism, the IP may never return, so there's now an invalid rating lowering the page's overall rating.
• There are no real rules or standards for rating
  • People interpret "power" differently, and some people even get the scale backwards. At a minimum, this should be changed to "balance".
  • Users rate their own work. Sometimes it is obvious, but sometimes there are borderline cases; what if someone just changes the wording? Or what if someone logs in as an IP?
• Ratings are not used anywhere. What's the point of rating if you can't sort by rating?

So those are the problems as I see them. Unfortunately, I don't also have much of a solution, either. The solution is probably a technical solution more than it is a creative one, so hopefully someone out there knows a thing or two about wiki add ins and can chime in. JazzMan 13:58, 7 July 2011 (MDT)

Jay Freedman 19:39, 7 July 2011 (MDT)

I know how to use the Rating System. The higher your Rating, the more PageViews you get! Hurray! ...But seriously. Jazz is totally right. The Rating thing is broken. It's just a popularity contest. And guys like me (The Almighty Jay), like to cheat at it. Because the Rating System doesn't work. It doesn't. It just shows how biased and lazy people are. ... "Uh, yeah man. I like this class. I'll give it a... Umm... 5 for power. And uh... How about a 6 for Formatting. Ya know. Cause the background color is still the same." ... Omg! Seriously! The background color! The whole website is that color! You can't even change it! Wow. ... Broken, broken, broken. Bleh. ...Thus! Solution: Get rid of it. And Don't replace it with anything. --Jay Freedman 19:38, 7 July 2011 (MDT) (Ding! Winning!)

Badger 18:48, 16 July 2011 (MDT)

I like the notion of ratings. However, I agree with everything said, and I don't have any solutions either. If I had my way, I would automatically delete any rating that still have the "remove this line" line. Beyond that, there doesn't seem to be any guideline as to rating a class. (If we come up with a decent system, I would be more than willing to write a guide on how to rate a class). Anyone have any thoughts on how we could better tie a rating system into the Featured Article system?

Jwguy 22:02, 16 July 2011 (MDT)

I agree entirely, and I can honestly say that I kind of made the mistake mentioned above, not knowing that the page rating was done manually. As far as solutions go, there was that big fiasco over a new ratings system, involving a ratings committee. That ended up with half the site's major contributors leaving after a huge clash and fallout with Green Dragon, who disagreed with the systen, though. Honestly, I kind of like the idea of a group of people dedicated to rating, forcing them to consider source and campaign specific material in their ratings, or lack thereof. I am not sure how well that'll go over, now, either, though. Unless the wiki has a function process mathematical data, and we can find a way to implement ratings into that, I couldn't say how to fix it.

Badger 22:44, 16 July 2011 (MDT)

I'm not sure that there is a way to automatically pull numbers out of the talk page, and then average them and move them to the actual page, in a box. I guess maybe we could do it with some sort of elaborate template, but I don't know anyone here who could code that (and leave it simple enough for casual users to implement it). As for the committee, I think that sounded like a good idea, but I happen to know the other wiki has since abandoned that concept. If I recall, it had to do with free time of committee members, and the ratings weren't as frequent as others would have liked. I think I'm content with just removing the rating system, and focusing on a way to enhance the way we work with featured articles.

Jwguy 02:46, 17 July 2011 (MDT)

Well, we have one option. This site's webshell is hosted, and paid for by Green Dragon, I assume, so I imagine we could petition him to use PHP or some other web language in order to make a function that could make ratings automatic. Granted, this requires us to find someone who will make the script, and Green Dragon to allow and implement it, which could be far-off shots, themselves. We'd also still have the matter of making sure people understand the rating system, while I am sure we could elaborate on this through the implementation of such a feature. (For example, by having a small paragraph with an example set of ratings and explanations, directly above the text field for the input of the rating, all in some small pop-up or another.

That said, I still believe the ratings committee would have worked. It's a good idea, but throughout the time constraints of members, something that would need to be considered in their eligibility for being on said committee, I believe it would have also helped if a system decided to alert and funnel articles in need of ratings were implemented, such as through a template that could be added by frequent users who understand when a page is ready to be rated.

Also, adding a section to the sorting table for classes and such for sortable ratings is easy. It can be done easily, I believe. The key problem that I see with their lack of use is, as stated before, it's either too convoluted to understand as a newbie, or the manual nature of the ratings.

  Hooper   talk    contribs    email   10:18, 17 July 2011 (MDT)

I believe that our current rating system keeps it as simple as possible (or at least I think that, but apparently 99/100 voters don't as they can't remove the giant remove line....). The idea of a pop up or something that easily and quickly explains it better would be best. Perhaps if a rating is added to a talk page we can create a script that automatically puts it into a category for admins or active editors to see what needs to have it's main page rating updated? Either way, its going to take some backend scripting or extension permission.

Rogue The Demonchild

An idea for rating: You could have a separation between in-progress classes and finished classes. In-progress classes can remain as they are now, but once someone deems their class "Finished" (being that they've actually gotten some critiques and improved it as much as they can), they can submit it as "Finished". The admins for this site (or maybe a small outside group of people that know how to rate classes) will be the only ones able to rate the "Finished" classes. [Also, any classes deemed "Finished" should be removed from the "Finished" list if the small group of raters have given it less than a 4/5 in any category; this is to make sure these classes actually can be considered finished].
This way, if a person wants to find classes that actually have a fully informed rating, they can click on "Finished Classes"; and if they just want to browse classes in general, they can click on "All Classes" or "Unfinished Classes".
Rogue The Demonchild 15:23, 15 April 2012 (MDT)

Green Dragon 19:01, 15 April 2012 (MDT)

That's not the problem. Making the rating system more complex is not going to solve anything. Is there a Semantic way of more quickly and simply adding a rating? Which script would do that? Or even, fixing the preload would help.

Jwguy 02:11, 16 April 2012 (MDT)

Yeah, something that handle the mathematics of the ratings and allow users to just input the numbers; That'd probably be best, as it'd make rating seem like less of a chore for those who know how, and make it actually work for those who don't know. The problem is, I feel like that would involve shell-access, anyways, because we'd have to have a database to draw the previous ratings from into the script, even if we found the aforementioned script. And that means database access and the like. I'm not that familiar with the inner workings of wikis, in general, so I'm not sure if there is any way to do so entirely within the wiki. I guess that's a research project. Jwguy 02:11, 16 April 2012 (MDT)

Jay Freedman 02:56, 16 April 2012 (MDT)

Ah yes. July 2011. Good month, good month. A bit old... But good. Anyway, still gonna with the whole 'throw it out' theory here. 'Featured Articles' is the modern way of 'preferring' pages. I'm sure we'd all love to see more articles that conform to being 'Featured' then articles that conform to being '20/20 something'. Think about it Green Dragon. More coding, more work, and more headache?.. Or just dump the Rating System and focus on getting more 'Featured Articles'. Quantity? Or Quality? ...Go for quality! Yippee! Save yourself the added work. (PS ...And ditch the Rating System all together.)

Ganre 15:29, 18 April 2012 (MDT)

I agree, the rating system doesn't work, but it is not because the system itself is flawed. Like most other computer problems, this lies squarely on the shoulders of the users. To counter this, i would suggest a system of checks and balances. People who are credible raters could flag a rating for review, and if it invalid for some reason (e.g. grossly misinterpreting the power score, or intentionally inflating the score). Flag it for removal.

Tivanir 16:02, 18 April 2012 (MDT)

So would these users be another subset kind of like admins and have their own group or would they be just a group of normal editors designated for this purpose?

Green Dragon 17:47, 18 April 2012 (MDT)

Making a new group of users is a bad idea. How do you do that with MediaWiki? Why the extra hassle? Why exclude users? Why be specific? Allowing users to rate is much more preferred. The question is, though, how do people rate best (I can tell you it is not as being part of a special group)?

Tivanir 18:19, 18 April 2012 (MDT) =

Well that was really my point I think. I don't know of any way to label someone as a "credible rater" and was really trying to figure out how they would be assigned. Granted I also accept the fact that many people have many separate ideas of how they view power (just off the top of my head there is core, all sourcebooks and homebrew as options) that can be used as baselines. The biggest thing I can think of to make it more useful would be to have a baseline class example for people who wish to rate classes, since then people could at least try to ballpark it.

Well, we could just do what we already do, and if we have a question on a rating, anyone can comment on it. If the question is significant, and is not answered, delete that portion of the rating. The only thing that would change is that we would incorporate this challenge system into the current policy. --Ganre 08:16, 19 April 2012 (MDT)

Ok so primarily instead of changing the system the idea is checks and balances, which does make more sense. Tivanir 10:40, 19 April 2012 (MDT)

Back to Main Page → Meta Pages → Discussions