Proposed ratings mechanism modifications
I think the post-rating concept has legs, although I'm a little unsure about the user interface to access them.
A Wiki-based post management system has some merit, but it'll take time and effort to massage a post into a suitable form. Posts are rarely standalones: they're usually replies to other posts, so the Content Editor(s) will need to trace the original conversation and distill it into the 'classic' post to provide any necessary context. It'll be interesting to see how this works out over the long term.
*
I've also been thinking about the original rationale for the ratings system. I'm one of those who has never paid any attention to my own rating. (Incidentally, it's never displayed in the standard post-reply page, so I won't even know what my rating is at the moment until I post this!) My problem with it is that it is content which matters, rather than the people who post it.
However, the value of content can often be subjective. Someone might be rated highly by a lot of newbies, while the few experts might spot the bullshit in it and try to counter those votes. Unsurprisingly, they'll often be outnumbered, so we end up with a Wikipedia-like situation where bullshit gets promoted to 'Gospel' status despite the best efforts of Subject Matter Experts.
I've seen similar situations here on GDNet, where a Expert Y chimes in (usually with his "grizzled, cynical bastard" hat on), points out that Poster X is talking out of his arse, only to get flamed because his post isn't 'polite'. I'm not someone to suffer fools gladly, and the few SMEs here are always going to be outnumbered by those who have far more time to kill, so an SME's accurate, albeit less than friendly, contribution may well end up ignored and result in the SME's rating being zapped. This is counterproductive: we need all the SMEs we can get.
I'd like to see:
1. A USENET "killfile"-type feature, whereby users can simply choose to hide posts by people they don't care about. The key user interface trick will be ensuring that even replies to those you've kill-filed get redacted accordingly. (I think this may be doable, although it's probably best left until the pressure is off and some time can be released for some pure R&D.)
2. If (1) is implemented, the current rating system is surplus to requirements and can be safely removed.
3. A more meritocratic system, whereby users perceived by mods and other staff members as being of value are awarded "Pillar Of The Community" status (or some such). SMEs could get an "SME" badge of some kind, and so on. These could either be voted on in a dedicated, public forum -- "Let whosoever believeth that Poster X is a fraud or tosspot speaketh his mind now!" -- or they can be handled behind the scenes, perhaps in a staff/mods-only forum, depending on the promotion being considered.
This is not to say that mere expertise gives one free rein to be a complete bastard to newbies -- mods can usually tell if a particular poster is popular, or otherwise, and act accordingly -- but it should remove the fear that (for example) an expert's imperfect grasp of English is going to be a barrier to acceptance. Willingness and ability to communicate knowledge clearly should be encouraged. At present, merely appealing to the masses is enough to raise one's ratings. You can achieve that without ever posting anything relevant to GDNet's core premise.
I wish there was a way to have Lounge-ed ratings separate from technical ratings. But oh well.
Quote:Original post by WashuQuote:Original post by Dave
I'm going to throw another spanner in the works and say what about if we some how restrict a users access depending on their rating, percentage or otherwise. For example:
<800 - 10 posts per day.
<500 - Only allowed in the lounge.
etc etc
Just things that would be annoying to them, eventually they wouldn't be allow to post at all if they get to <100 or 0 or something. It's a self moderation system governed by the users of the site.
Dave
That's stupid. Sometimes, on a rare basis, people with ratings less than 100 make it back up. And forcing someone with a rating less than 500 to ONLY post in the lounge WILL reduce them to a rating of 0. The ratings, as they stand have NO bearing on the moderation of a user. It will always be that way. However, the proposed changes the Oluseyi has mentioned would enable us to easily identify users who are helpful, and content that is useful.
Yeah i totally agree.
Is there any way that we can moderate users without the moderators/staff? Determining that a troll is useless doesn't stop him being useless. If we could get rid of the trolls we could enrich the content of the site.
Author
Quote:Original post by coderx75
As for archiving, flagging threads as favorites (which already exists) would keep them from being archived/deleted.
People have a lot of garbage favorites. People often forget to purge bookmarked favorites. Saving a thread because just one person counts it useful is questionable, from a raw efficiency perspective, and if there is a readily searchable and browseable record of distilled information, it may simply be wasteful.
Quote:Original post by Palidine
What's the motivation behind the change? Is it just a desire to save on operating costs by limiting database size? Is it just a cleanliness thing: maintain a database with a higher density of good information?
Those are both considerations, though by no means the only ones. I'm not at liberty to discuss all the considerations, but DB size is a huge concern.
Admittedly, that concern can be somewhat mitigated by better caching and more efficient query construction/execution.
Quote:If it's anything other than a cost reason I say keep the data. Nothing is worse to me than the destruction of knowledge; what's to say that something which doesn't meet the classic post bar for the editors isn't immensely useful to someone who hasn't created an account yet.
We are to say. But, please, spend some quality time trawling through five year old posts and tell me the incident rate of coming across a truly non-redundant gem (ie, knowledge that has not since been repeated).
In the short term, we will lose access to such valuable data if we implement the new strategy with a hard cut off, which is why we'll archive all the old (current) content for a minimum twelve month period, during which, with improved search, we should be able to get a feel for what content is a preservation candidate.
Quote:My gut instinct is to stay with the unbounded points system (only because it provides the same satisfaction for me as gaining levels in an RPG). A capped system would, on first inspection, make me look at my rating more as something that must be maintained rather than a fun continual improvement loop.
I'm not terribly particular about that, so feel free to continue to make the case for your preferences. [smile]
Quote:I definitely otherwise like the classic post & classic thread system as a different mechanism by which data can be accessed. It'd be awesome to have a classics section just open for user viewing; that way when you're bored you can just start browsing and learn cool stuff.
Cool.
Quote:Original post by smitty1276
Are you sure that there actually is enough content that is effectively "dead weight" that it would outweigh the overhead of having potentially tens of thousands of unique user ratings applied to tens of thousands of threads and posts?
Yes, there is sufficient dead weight. The aim of this system is to build in structural checks to minimize the creation (or expedite the pruning) of dead weight in the future. Further, once a post has been "classic'd", no further ratings against it will actually be counted (though we might leave the UI there, for the diabolical joy of having latecomers click fruitlessly... [evil]).
Quote:Original post by arithma
- The rating system's purpose must be CLEARLY defined and stated.
Unlikely, given that even the staff and mods are torn about what, exactly, the user ratings system's purpose is.
Quote:- One can nominate a post to be in a FAQ-like section (the community sticky?) in the sub-forum (i.e. General Programming).
Nah. Allowing any member to nominate posts would create too much work for the mods - some malicious users would nominate every random piece of crap, much like some users currently flag every little argument they have in the Lounge (you know who you are, and, more importantly, I know who you are).
Quote:- A mechanism for the post to actually go into the community-sticky section must be devised based on votes from users (taking into account their rating)
Again, unlikely. The mechanism for nomination would be devised based on user votes, but actual acceptance would be an executive decision. User representation in that executive process would be increased by the Content Editor roles being much more open than moderator/News Editor/Staff.
Quote:- Viewing in the community-sticky section should allow the user to sort according to rating or according to time (to solve "dusty posts" problem).
Post ratings would disappear once accepted into the classic area.
Quote:Original post by Nytegard
I guess my area of concern about this comes to, what makes a classic post?
Fear not. Content Editors, Moderators, Staff and other "people with useful tags" will be able to directly nominate (but not accept) posts into classic status.
Quote:Original post by Dave
I'm going to throw another spanner in the works and say what about if we some how restrict a users access depending on their rating, percentage or otherwise.
No.
Quote:Original post by Alpha_ProgDes
Well would my post rating affect my user rating?
Your post rating?
The rating on a post is fundamentally distinct from an overall user rating. I haven't decided - or even initiated discussion - on whether post ratings should influence user rating. They probably should, though how remains to be determined.
Quote:Also, will the ratings (for posts, threads, and articles) be numerical, labelled (ie. trash, bad, average, good, excellent), and/or some sort of colored graph/chart?
Probably textually labeled, with numerical equivalents.
Author
Quote:Original post by Dave
Is there any way that we can moderate users without the moderators/staff? Determining that a troll is useless doesn't stop him being useless. If we could get rid of the trolls we could enrich the content of the site.
I don't see that as a good idea. I think the richest tool we can give you is the option of a killfile, where you simply don't see posts made by a user. (You will see, however, that user's posts when quoted by someone else. Getting around that is just too much work.)
Author
Quote:Original post by stimarco
A Wiki-based post management system has some merit, but it'll take time and effort to massage a post into a suitable form. Posts are rarely standalones: they're usually replies to other posts, so the Content Editor(s) will need to trace the original conversation and distill it into the 'classic' post to provide any necessary context. It'll be interesting to see how this works out over the long term.
Indeed. This is why I propose simultaneously expanding the number of content editors and requiring a minimum of two votes for final acceptance - that way at least two pairs of eyes will have cursorily looked a thread over. Plus, we can archive the thread temporarily while revisions are made to the post, giving the content editors the opportunity to increase quality over a duration instead of as a single shot.
Quote:1. A USENET "killfile"-type feature, whereby users can simply choose to hide posts by people they don't care about. The key user interface trick will be ensuring that even replies to those you've kill-filed get redacted accordingly. (I think this may be doable, although it's probably best left until the pressure is off and some time can be released for some pure R&D.)
Interesting idea. Probably doable.
Quote:2. If (1) is implemented, the current rating system is surplus to requirements and can be safely removed.
Maybe, but where's the fun in that? [smile]
Quote:3. A more meritocratic system, whereby users perceived by mods and other staff members as being of value are awarded "Pillar Of The Community" status (or some such). SMEs could get an "SME" badge of some kind, and so on. These could either be voted on in a dedicated, public forum -- "Let whosoever believeth that Poster X is a fraud or tosspot speaketh his mind now!" -- or they can be handled behind the scenes, perhaps in a staff/mods-only forum, depending on the promotion being considered.
The idea of "merit badges" has been floated repeatedly and is generally agreed to be a sound one.
Quote:Original post by OluseyiQuote:Original post by stimarco
A Wiki-based post management system has some merit, but it'll take time and effort to massage a post into a suitable form. Posts are rarely standalones: they're usually replies to other posts, so the Content Editor(s) will need to trace the original conversation and distill it into the 'classic' post to provide any necessary context. It'll be interesting to see how this works out over the long term.
Indeed. This is why I propose simultaneously expanding the number of content editors and requiring a minimum of two votes for final acceptance - that way at least two pairs of eyes will have cursorily looked a thread over. Plus, we can archive the thread temporarily while revisions are made to the post, giving the content editors the opportunity to increase quality over a duration instead of as a single shot.
It'll end in beers. Mark my words...
Quote:Quote:1. A USENET "killfile"-type feature, whereby users can simply choose to hide posts by people they don't care about. The key user interface trick will be ensuring that even replies to those you've kill-filed get redacted accordingly. (I think this may be doable, although it's probably best left until the pressure is off and some time can be released for some pure R&D.)
Interesting idea. Probably doable.
It occurs to me that adding this feature could make for some really odd conversations. Perhaps a 'threaded' view option for forum posts might be necessary. Make it an optional feature. (I do miss being able to follow threads and branches in the old USENET fashion. But then, I'm old enough to remember Spry's Mosaic browser and the mercifully brief fad for the < blink > tag.)
Quote:Quote:2. If (1) is implemented, the current rating system is surplus to requirements and can be safely removed.
Maybe, but where's the fun in that? [smile]
In that case, might I suggest making it a bit more... fun?
An RPG-like metagame could be created quite easily: give each (non-Anonymous) poster a "Trump Card"-type ident next to each post. (With the rise of widescreen displays, this may be more efficient than the current UI, which doesn't take advantage of the trend.)
This would show RPG-like stats for each poster...
- If a poster is proving really useful, other posters can increment his "Social Worker" rating.
- Older posters can have a "Grizzle Factor" which increases with age (ideally displayed using a suitable "beard" graphic).
- Talkative posters would score highly on the "Verbosity Scale"
- ... and so on.
The 'card' image and stats could be either numeric, or graphical, or a combination of both. GDNet+ subscribers would be able to pick and choose some variant graphics, such as the central image, etc.
I recall a "GDNet card game" thread that touched on a similar subject some time ago which may be worth mining for ideas. I'm sure others will be able to add other perfectly silly ideas to the pot.
It'd certainly make GDNet's forums stand out from the crowd, as well as providing an interesting example of meta-game design that would make ratings far more fun. Fun is, after all, what games are supposed to be about...
Quote:Quote:3. A more meritocratic system, whereby users perceived by mods and other staff members as being of value are awarded "Pillar Of The Community" status (or some such). SMEs could get an "SME" badge of some kind, and so on. These could either be voted on in a dedicated, public forum -- "Let whosoever believeth that Poster X is a fraud or tosspot speaketh his mind now!" -- or they can be handled behind the scenes, perhaps in a staff/mods-only forum, depending on the promotion being considered.
The idea of "merit badges" has been floated repeatedly and is generally agreed to be a sound one.
This could be tied in to the 'trump card' idea above, with all the badges and stuff appearing in the area to the left of the post's text. Since the world+dog seems to be moving towards widescreen aspect ratios, favouring a wider UI makes sense.
Anyway, I'm off to bed.
I've thought about my position a bit more.
Per post ratings are probably better. There are many times when I'm skimming a thread and right now I tend to look at names and ratings to judge whether I should read a post. For example, I'm skimming a Win32 thread I tend to look for Endurion and Lessbread since they both tend to have detailed and informative posts. I have probably missed some good information doing this. If there was a way that users could mark posts as helpful that would help me out while skimming.
I do see the scenario that many good posts wont get marked as such unless there is a very easy way to do it. It would have to be inline like Amazon's rating system because nobody is going to want to go to another page to rate a single post. Even then, if most people dont tend to use it how big of a success could it be? Per post ratings seem to really depend on the continued participation of a lot of people whereas the per user ratings only depend on occasional participation. I can go a long time before I see somebody do something that I consider worthy of rating them and I'm sure I'm not alone. This doesnt seem conducive to your much more finely grained system.
And stimarco, your RPG-style infographics do seem like a fun idea, although probably more as an April Fools amusement. It might be a little much in the normal forums.
Per post ratings are probably better. There are many times when I'm skimming a thread and right now I tend to look at names and ratings to judge whether I should read a post. For example, I'm skimming a Win32 thread I tend to look for Endurion and Lessbread since they both tend to have detailed and informative posts. I have probably missed some good information doing this. If there was a way that users could mark posts as helpful that would help me out while skimming.
I do see the scenario that many good posts wont get marked as such unless there is a very easy way to do it. It would have to be inline like Amazon's rating system because nobody is going to want to go to another page to rate a single post. Even then, if most people dont tend to use it how big of a success could it be? Per post ratings seem to really depend on the continued participation of a lot of people whereas the per user ratings only depend on occasional participation. I can go a long time before I see somebody do something that I consider worthy of rating them and I'm sure I'm not alone. This doesnt seem conducive to your much more finely grained system.
And stimarco, your RPG-style infographics do seem like a fun idea, although probably more as an April Fools amusement. It might be a little much in the normal forums.
Instead of collecting all of the "classic/informative" threads into one big collection, could it be instead possible to have each forum list at the top "threads of the week"/"threads of the month"(based on post rankings). This way people who haven't visited the forums for a while can immidiately see the most important/informative threads in the recent past.
Quote:Original post by Oluseyi
These are extremely preliminary and not at all official! What do you guys think?
Ah, good, so I can speak freely on this [smile]
Quote:Note that this is entirely independent of the other proposed points. Building extra data about forum posts - specifically the 'value' of a forum post - can be used in a great many ways, from a "classic posts" gallery, to improving search results, to generating a forum FAQ, to calculating user ratings. It's actually a perfectly decent feature without anything built on top of it - people will like to see their posts get marked as helpful, even if those marks are not actually used for anything else.
Individual user posts can be rated up/down for their contribution to the community.
Quote:Sufficiently highly rated posts are automatically nominated for "permanence" - addition to a long-term record of "classic posts" containing particularly informative/helpful/entertaining information.I don't like this - you're mixing up a feature with our post deletion policy (which to date has been 'archive everything'). I'd drop the 'classic posts' idea and just stick with the "things that aren't helpful are liable to get deleted" policy. Also, if we fix things like the search then deletions need only be driven by hardware/software requirements.
Quote:User ratings are normalized aggregations based on post ratings. Instead of unbounded numbers (how high can current ratings go?), they'll instead be a percentage which could be considered analogous to presidential approval ratings.Not sure about the best way to approach this - if it happens then we need to reconsider the way in which such values are displayed. Should they be displayed on each post? Should they be visible to everyone or just the user themselves? etc etc...
This topic is closed to new replies.
Advertisement
Popular Topics
Advertisement
