Jump to content
  • Advertisement
Sign in to follow this  
Oluseyi

Proposed ratings mechanism modifications

This topic is 4460 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Quote:
Original post by Oluseyi
Quote:
Original post by ApochPiQ
I like the idea, with one tweak - there should be some kind of per-user weighting to the post ratings. Zahlman giving a post 5 stars should mean a hell of a lot more than, say, me, or JoeNewbie who joined yesterday.

This goes back to the problem of interpreting rating as a measure of expertise. boolean is the tenth highest rated non-mod, but I don't put much weight on his technical skills, personally, so why should his rating of a post as helpful mean more than S1CA?

I'm not rejecting the idea. I'm just not convinced yet.

Well we already having ratings and supposedly we're going to get badges. So why not let the two determined the weight of the post rating? ApochPiQ has 1800 and gives a post 5 stars obviously that's different from Zahlman who has a badge of "SE Guru" and a rating of 1800. So Zahlman's rating of 5 stars would hold more weight than ApochPiQ's. Obviously, I'm assuming that user ratings will stay and badges have a rank or weight value associated with them.

[Edited by - Alpha_ProgDes on March 22, 2007 4:06:40 PM]

Share this post


Link to post
Share on other sites
Advertisement
Quote:
Original post by Oluseyi
This goes back to the problem of interpreting rating as a measure of expertise. boolean is the tenth highest rated non-mod, but I don't put much weight on his technical skills, personally, so why should his rating of a post as helpful mean more than S1CA?

I'm not rejecting the idea. I'm just not convinced yet.



That's precisely what I'm hoping can be cured, although I didn't say it too well.

Like I said, I have no real ideas for how to actually implement such a thing; certainly a simple calculation based on, say, current user rating isn't going to work well.


I think the question partly comes down to purpose. Are we interested in flagging technically valuable content, or just "nice posts" from "nice people"? Judging from the suggestions of archiving highly rated posts, I assume that the intent here is to focus more on the technical value side of things. (I mean specifically that the per-post rating is meant to have this focus; not necessarily the site/community as a whole.)

As such, if a per-post rating (or at least some part of the per-post rating system) is specifically emphasized as a metric of technical merit, it should be possible to aggregate a user's post-rankings to construct the weighting value. However, as long as there is any component of mere "helpful" or "friendly" or "funny" or whatever, that function is clearly disrupted.

The current rating system was intended to be a self-moderation tool; it's designed to measure how well people play nice and get along. For that purpose I think it's worked quite well.

However, we have a separate need in the community, and that is to gauge technical value. This is, by and large, completely orthogonal to measuring community participation skill - hence the common problem of figuring out if someone's high rank is because they're making an effort to not be a dick (like me), or because they actually know something and are worth listening to.


I think both metrics are needed, but clearly any solution that tries to represent both concepts in a single number is not going to be optimal. I'm not sure I like the idea of muddying the waters by attaching two numbers to everybody, but it may be unavoidable, and with suitable interface-level abstraction it should be manageable.

Share this post


Link to post
Share on other sites
I like the idea of per post ratings but...

With regard to per post ratings, a five star system suggests only positive ratings. That seems fine for noting outstanding posts, but it would seem to disallow giving negative ratings to posts that deserve them. It would in effect equate posts with a no star rating with posts that merited a negative five star rating. And if posts can be rated negatively and edited, it seems to me that posts with negative ratings would subsequently end up edited for content. I like being able to edit posts. Aside from the obvious spell checks and url fixing, sometimes it's better to add an afterthought to a post than to add a new post and push the thread back up to the top of the active list.

Share this post


Link to post
Share on other sites
I like the idea of a killfile, where I could have GameDev not show me posts by certain users. I would jump on this in a heartbeat, and I think my overall experience of GDnet would jump up as well. At the same time, the ability to click a button on the thread view to show the post anyway would be nice for the rare occasion that <name redacted> had to say.

I like the badge idea. For example, Professor420 deserves some real credit for his posts, even though many people flame him for having a brusque manner of giving advice. Incidentally, this would allow S1CA and boolean to receive credit for their achievements, though they be in different areas.

I like being able to rate individual posts, but not some of the other changes.

I don't like not archiving the fora. First, it would mean a massive, massive effort would be necessary to deal with the millions of posts already made, even though new posts would not be so much of a hassle. I might venture that there is a high incidence, if not prevalence, of useful posts already made.

It would also require a lot of effort to make classic threads into wiki-type articles. Additionally, there might not be many people capable of handling many of the threads, and this might put too much of the responsibility on a very few users.

I'm okay with linking post and user ratings, but not necessarily as described in option 3.

I agree with LB's post above mine.

Share this post


Link to post
Share on other sites
Quote:
Original post by The Reindeer Effect
...I hold that the merit of a post (at least, in technical discussions) is due to the content of that specific post, not the amortized popularity of all posts made by the poster (especially considering that you can rate people in lounge posts)...


On a slightly related line of thought.
Could we re-allow anonymous posting in the Graphics Programming and Math forums?
Well... Graphics Programming at the least? I can understand that Math might be more rigorous so you'd want to enforce names there...

Share this post


Link to post
Share on other sites
Quote:
Original post by SuperPig
...

Incidentally, there's nothing that says per-post ratings have to replace the existing per-user rating. The two could coexist, at least temporarily.


I'll agree to that. I've been skimming through the thread thinking I actually rather like the current system as it is.

Sure, post-based ratings may provide more useful data for topic classification and may even be more suitable for aggregating user ratings than the current system. But as long as we keep seeing 50+ XNA vs. MDX vs. C# vs. C++ vs. OpenGL vs. ATI vs. NVidia vs. MMORPG topics a week, I doubt the post rating thing works out without a decent (independent?) user rating to go with it, to have a means of weighing the judgement of a post-rater.

Share this post


Link to post
Share on other sites
incidentally, i like the idea to some extent. However people are only going to rate a post if it's helpful to them at a specific point in time. You are only going to find it helpful if you're looking for some information, so you need a way of arriving at the helpful post - which is currently google. Adding a tagging system to threads would be beneficial, especially if it's autosensing or if we exploited SQL Server's Fulltext catalogue abilities.

Also bear in mind that boolean did something similar with his help portal which started taking off until he had an issue with the hosted site that never got resolved. You might want to take his code and merge it into the GDNet system. You know, save reinventing the wheel and all that...

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!