Jump to content

  • Log In with Google      Sign In   
  • Create Account


Reputation system easily abused


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
13 replies to this topic

#1 wodinoneeye   Members   -  Reputation: 618

Posted 12 October 2012 - 05:53 PM

Example - someone repeatedly marking down rating (-3 each ?) for whatever petty reason. every posting made by a user in a thread - there many be alot of back and forth discussions making it easy to do this.

So fairly easily just one person could significantly drive down any rating score of anyone who posts alot here (can seek out all postings to a user name and go thru all of them to mark down if they have nothing better to do).

Imagine a Moderator who posts alot having every posting theyve ever made marked down (assuming thats not blocked in the ruleset) -- how easy it would be to make whatever REAL rating they have irrelevant.- driving it to zero with not much effort

I dont know if the forum system is changeable to any extent to fix something like this. Assuming thats not a problem - What change could be made would be the issue (ex- only one down mark per thread per posting user maybe or having the down/up rater have to provide a reason text input or a minimum time limit between markings)


-- It is the usual problem of having to nerf the system because just a few nitwiits can easily abuse it.
--------------------------------------------Ratings are Opinion, not Fact

Sponsor:

#2 Servant of the Lord   Crossbones+   -  Reputation: 14852

Posted 12 October 2012 - 06:59 PM

(I'm not a moderator, just so it's clear)

You forgot to mention that you were the user being downvoted. That's pertinent information.
You also forgot to link to the thread in question: AI Creation

Do you know:
A) Every time you rate someone down (costing them 3 points), it also costs you 3 points? They sabotage their own rating by sprees of down-votes.
B) Users with higher ratings have more effect than users with lower ratings? Someone downvoting sabotages their own vote-weight by rating negatively too much.
C) You can click on the number between the Up and Down arrows, and see who rated the posts down? They aren't anonymous.

That helps balance the system somewhat.

Edited by Servant of the Lord, 12 October 2012 - 06:59 PM.

It's perfectly fine to abbreviate my username to 'Servant' rather than copy+pasting it all the time.

[Fly with me on Twitter] [Google+] [My broken website]

All glory be to the Man at the right hand... On David's throne the King will reign, and the Government will rest upon His shoulders. All the earth will see the salvation of God.                                                                                                                                                       [Need free cloud storage? I personally like DropBox]

Of Stranger Flames - [indie turn-based rpg set in a para-historical French colony] | Indie RPG development journal


#3 Bacterius   Crossbones+   -  Reputation: 7001

Posted 12 October 2012 - 07:17 PM

I agree with Servant of the Lord, although I have to correct some points you made:

A) Every time you rate someone down (costing them 3 points), it also costs you 3 points? They sabotage their own rating by sprees of down-votes.

Actually, only 1.

C) You can click on the number between the Up and Down arrows, and see who rated the posts down? They aren't anonymous.

Huh. I can't click it. Can anyone confirm this? I recall this being a moderator or GDNet+ feature..

"The best comment is a deleted comment."


#4 wodinoneeye   Members   -  Reputation: 618

Posted 12 October 2012 - 08:10 PM

(I'm not a moderator, just so it's clear)

You forgot to mention that you were the user being downvoted. That's pertinent information.
You also forgot to link to the thread in question: AI Creation


I didnt mention it because that kind of thing just encourages certain people to do that behavior (that it matters to me and thus 'they got me' )

whereas Im more interested in how systems work or dont work (and what solutions might exist)

I could care less about MY rating in a system that is so easily abused (the old system I expressed opinions in the 'lounge' once and watched some petty people (a clique actually, who have problems when other people express ideas different from their own) there drive down my rating alot --wasnt even a technical discussion having to do with games (but there is no way to seperate ratings there from the game related topics - short of a seperate rating system for each forum and even then some obsessed nitwit could trace and down your user elsewhere - unless you had a different identity that wasnt linked)

SO after that and some talk to a mod where he just said he stays away from the 'lounge' and the type of people who have mental issues there -- I simply discounted the whole rating system as irrelevant.

Edited by wodinoneeye, 12 October 2012 - 08:39 PM.

--------------------------------------------Ratings are Opinion, not Fact

#5 wodinoneeye   Members   -  Reputation: 618

Posted 12 October 2012 - 08:20 PM

I agree with Servant of the Lord, although I have to correct some points you made:


A) Every time you rate someone down (costing them 3 points), it also costs you 3 points? They sabotage their own rating by sprees of down-votes.

Actually, only 1.

C) You can click on the number between the Up and Down arrows, and see who rated the posts down? They aren't anonymous.

Huh. I can't click it. Can anyone confirm this? I recall this being a moderator or GDNet+ feature..



Are these rating rules spelled out anywhere ??

A -1 on your own rating for issuing a 'down' rating at least would be some repercussions (maybe block further markings once YOU reached 0 )
(Im not sure what kind of IP checking for multiple accounts there is -- because shill accounts could be another loophole used by the obsessed )

Is there a similar +1 for rating someone up (that could be another loophole to distort any real 'rating' statistics)

A limt per day ... etc ....


The old system would not show who down rated you (only up) I guess to prevent reprisals


I noticed the current 'easy abuse' when I saw -3 a bunch of times all logged for the same minute.

The only relevance (real impact) I can think of is if people filter postings by rating but I dont think any significant number of people here do that (or even know there is such a feature)

-----

experiments

yes your rating DOES go up when you you up someone else

you get "You have reached your quota of positive votes for the day" after a handful

Edited by wodinoneeye, 12 October 2012 - 08:40 PM.

--------------------------------------------Ratings are Opinion, not Fact

#6 Servant of the Lord   Crossbones+   -  Reputation: 14852

Posted 12 October 2012 - 09:15 PM

I agree with Servant of the Lord, although I have to correct some points you made:


A) Every time you rate someone down (costing them 3 points), it also costs you 3 points? They sabotage their own rating by sprees of down-votes.

Actually, only 1.

Ah, thanks! I've only found the need to downrate people a few times since the new system has rolled out, so I wasn't sure.

C) You can click on the number between the Up and Down arrows, and see who rated the posts down? They aren't anonymous.

Huh. I can't click it. Can anyone confirm this? I recall this being a moderator or GDNet+ feature..

Whoops, probably GDNet+. Posted Image
Still, they aren't anonymous to alot of users, and when abused or given unfairly, I've noticed that others upvote to compensate and I also do the same.

Are these rating rules spelled out anywhere ??

Not that I know of, but these rules are in effect. I know when I rate up people, I do +4 (though a bug lists it as +3). I know when others with high ratings rate me up, the give me +4 (though it says +3). I think it's capped at +4 though - seems like a pretty weird formula.

There are checks and balances in place - but the biggest check and balance is the community itself. When things are actively being abused, sometimes a moderator might roll those ratings back and warn the user, but mostly the community itself steps in silently to undo the damage. At least, this is what I've personally noticed and personally done - though you can't always see it if all you see is a +0 or -1.

Someone already undid the negative rating for one of your posts, by countering it with a positive rating of their own. As more people see the thread, they'd either think it contributes and uprate, or think it doesn't benefit the thread and downrate. Ideally. Yea, some rate based off of "like/dislike" or "agree/disagree", but most do it right, and it balances out surprisingly quickly and lets helpful users zoom upwards, and rude/inaccurate/spamming users zoom downward, depending on their frequency of posting and the quality of the content their posts contain.

My rating isn't high because my individual posts are "more helpful" than other peoples', but because I procrastinate more on these forums than others (Posted Image), and my procrastinations are directly beneficial to numerous users (so I have more posts that are helpful, rather than each individual post being "more helpful" that someone else's helpful post). The rating system encourages frequency and quality of helpfulness, and discourages frequency and intensity of rudeness and non-constructive debate - which is exactly what a community needs to thrive (active users that are also helpful users, as long as they are knowledgeable users).

Hopefully this makes sense?

Is there a similar +1 for rating someone up (that could be another loophole to distort any real 'rating' statistics)

Yes, you get +1 for rating someone up, but your +1s for rating someone else up, is rapidly outpaced by your +3s of being rated up if you actively help others.

The short-term benefits or detriments of abusing the system fade out in the long-term effects of the community. Sometimes I get rated down for not giving the answer a user wants even if my post is right. Their -3 is rapidly washed over by ten times as many +3s by other members saying, "Yes, his answer is correct or helpful", either in the same thread or other threads. Any abuse I've encountered (of which I've encountered shockingly little for such an active forums) is compensated for very rapidly.

It's perfectly fine to abbreviate my username to 'Servant' rather than copy+pasting it all the time.

[Fly with me on Twitter] [Google+] [My broken website]

All glory be to the Man at the right hand... On David's throne the King will reign, and the Government will rest upon His shoulders. All the earth will see the salvation of God.                                                                                                                                                       [Need free cloud storage? I personally like DropBox]

Of Stranger Flames - [indie turn-based rpg set in a para-historical French colony] | Indie RPG development journal


#7 jbadams   Senior Staff   -  Reputation: 14699

Posted 12 October 2012 - 09:31 PM

There's a description of how the system functions in the staff journal.

Some notes on the system:

When you up-vote another user's posts you also gain a point yourself as a reward for participating in the system. I haven't seen more recent statistics, but if I recall correctly in the first week it was implemented participation in the system was increased about 400%. As a balancing measure however, there is a limit (I think it might be 5 times per day?) on how many times you will be rewarded for up-voting each day; note that you can still up-vote more posts and it will still effect the other user's reputation, but you won't be rewarded.

When you down-vote another user's posts you sacrifice one point of your own reputation. This is to discourage frivolous down-voting and provide something of a natural counter to abuse.

The effect of votes is weighted based upon the reputation of the voting user. Votes from a higher rated user have a greater effect on the reputation of the target user than votes from a lower-rated user. I'm not sure of the exact threshold, but the votes of users with a low reputation won't have any effect on other user's reputation. They can still vote on posts but it won't have any actual effect.


Many of our users tend to vote towards what they think is a fair value, thereby helping to self-moderate the system; if they see a post that has been down-voted and do not think the down-vote is fair they will balance it out with their own up-vote even if they might not have otherwise voted at all, and vice versa. In the vast majority of cases this simple self-moderation on the part of our community tends to balance out any small instances of unfairness, whilst moderator intervention can handle more systematic abuses.


There's also an upper limit on how many total votes can be given by any user in a single day, but we have it set relatively high so as not to interfere with legitimate usage of the system. We do however keep an eye on the system and are willing to investigate if someone reports a possible abuse; there have been a few cases where we've discovered abuses of the system and manually rolled back the effects of the votes in question. I'll add checking your reputation (and that topic in particular) to my to-do list and try to get to it within the next few days.

#8 wodinoneeye   Members   -  Reputation: 618

Posted 12 October 2012 - 09:36 PM

I recall the incident that apalled me (years ago) was a posting in 'the lounge' (I recall some opinion about use of a-bomb in WW2, motives/reason that wouldnt raise an eyebrow in a real politics oriented forum ) that in one day drove my rating down 150-200 points (which I had earned by many helpful posting to beginners forum and some from the more contentious game design forum) seemingly disproportionately negative (for one posting of opinion and some qualifiing answer posts) --- all in a subject having nothing to do with game development.

I though if I wanted to 'get even' I could create a hundred shill accounts and down mark like crazy - automate it even -- but then eh that would simply be a waste of my time with such petty disturbed people (intelligence in one area can be offset by stupidity and foolishness in others....). SO I ignored the dimwits and their petty games (and hence my sig line thats been there now for years)



"The effect of votes is weighted based upon the reputation of the voting user. Votes from a higher rated user have a greater effect on the reputation of the target user than votes from a lower-rated user. I'm not sure of the exact threshold, but the votes of users with a low reputation won't have any effect on other user's reputation. They can still vote on posts but it won't have any actual effect."

that might of been the effect years ago ( a few high rateds - rating each other up for all I know gouged my rating --- heh they must feel soooo powerful heheh) -- but that itself might be a major distortion as why should expertise/acclaim in one area neccessarily translate into MORE rating influence in another ??



"but the votes of users with a low reputation won't have any effect on other user's reputation"

No effect ?? So they also damage and invalidate the downgraded persons ability to rate ?? Another potential defect/dispropotion.


Its like in so many online games the griefers/abusers cause the game to be nerfed into patheticness.



"I'll add checking your reputation (and that topic in particular) to my to-do list"

its no big deal, Im more interested in the rating system's design than any effect it might have on any rating I might have (before playing around with it this last day I dont think I even rated anybody or that much paid attention to the number). For me ,what postings say actually being the importatnt thing and a realization that not everyone thinks the same way or can even present what they want to say clearly

.
--------

Similar problems exist with any complaint system in MMORPGs where any reporting mechanism (particularly largely automated ones) are subject to just more destructive griefer loopholer's behavior (just another channel to be turned against normal players) and any effective enforcement really requires actual GM interventions, which game companies are generally loath and disinclined to pay for..

Edited by wodinoneeye, 12 October 2012 - 10:03 PM.

--------------------------------------------Ratings are Opinion, not Fact

#9 jbadams   Senior Staff   -  Reputation: 14699

Posted 12 October 2012 - 10:31 PM

A lot of users objected to losing reputation because of posts in the lounge, and we agreed that this wasn't fair. You might notice voting is no longer enabled in our off-topic forums such as "The Lounge" and "Comments, Suggestions & Ideas".


Users also wanted votes to be per-post rather than per-user so they could see more precisely what was being rewarded or objected to, which I think has been a great improvement over the old system.

The increased effect of higher rated users are now also much more limited than they used to be; I believe the highest possible effect of a vote is +/-5 votes, rather than around 100 points of change from a single vote under the old system.


"but the votes of users with a low reputation won't have any effect on other user's reputation"

No effect ?? So they also damage and invalidate the downgraded persons ability to rate ?? Another potential defect/dispropotion.

It's a very low threshold -- I'm not sure of the exact value, but I don't think it kicks in unless you reach 0 or a negative value. Given all users are given a starting value of 100 points it takes a reasonable number of down-votes before this effects you, so I don't think it unfairly sensors anyone -- just logging in occasionally (you can earn 1 "participation" point per day simply by logging in, or visiting the site if it's remembering your login) -- keeps most users from losing the ability to effect reputation.

The chance of one person -- or a small group -- being able to inflict this upon another user unfairly are pretty slim, and almost sure to be noticed and reversed. I only personally know of one user this effect actually applies to, and it's definitely a good thing that his votes don't count.


"I'll add checking your reputation (and that topic in particular) to my to-do list"

its no big deal, Im more interested in the rating system's design than any effect it might have on any rating I might have (before playing around with it this last day I dont think I even rated anybody or that much paid attention to the number). For me ,what postings say actually being the importatnt thing and a realization that not everyone thinks the same way or can even present what they want to say clearly

Fantastic attitude to have! I will still be investigating however, both out of fairness to your score as well as to fix any impact on the score of whomever has down-voted you.

#10 Shippou   Members   -  Reputation: 1089

Posted 13 October 2012 - 03:10 PM

Folks routinely get down voted for having a newbie score ( I've seen this a lot ).
Folks usually get down voted by saying anything negative about almost any programming language ( fan boys ).
Folks get down voted quite often for not recommending someone else's favorite language to beginners.

It boils down to having to figure out the exact format to write one's post, based on what is populate that day, and who is online. I've stopped posting in "point" areas, due to the "mob mentality" being really counter productive.

My YouTube Channel Filled With Geek, Nerd, Politics, Economics, & More !  
[Click Here]


#11 Gaiiden   Senior Staff   -  Reputation: 4282

Posted 13 October 2012 - 04:53 PM

There's no perfect system, and if anyone feels they are being abused via the rating system we always take the time to look into it and take any corrective actions necessary so long as we're made aware of the issue.

Drew Sikora
Executive Producer
GameDev.net


#12 MrDaaark   Members   -  Reputation: 3535

Posted 13 October 2012 - 05:19 PM

It boils down to having to figure out the exact format to write one's post, based on what is populate that day, and who is online. I've stopped posting in "point" areas, due to the "mob mentality" being really counter productive.

I haven't seen this anywhere here.

Believe it or not, some people do give bad advice, argue just to argue, or make crap posts. The down vote on bad advice is not a judgement on the personal worth of the poster. Obviously, the poster had good intentions with their advice, but that doesn't change that they aren't posting a good answer. So it becomes noise, and it helps the person with less experience realize that it wasn't a good answer. We rate posts, not users.

Awhile ago, someone asked a simple question and some idiot crashed the thread with an elitist response that wasn't even appropriate to the situation. Then he tried to justify it by going on long rants about what does and doesn't work in big budget AAA scenarios, when no one was talking about that, and kept going, and wouldn't shut up despite being off topic, and out of context. So he took a nice beating with down votes. It was a nice reminder that his attitude is not needed or required here, and the beginning user could find the best advice.

#13 samoth   Crossbones+   -  Reputation: 4066

Posted 13 October 2012 - 05:41 PM

Looking at that thread, it seems to me that you upset someone by insisting after several people who have done game AI work told you (several times) that it doesn't work that way and doesn't make sense in practice.

Though I think that even being told "dude, what you're saying is total nonsense, it really works like this..." may be useful to a reader (who might have a similar idea). Insofar, I don't see your posts as so terribly bad that they deserve a downvotes, at least not the first ones (which I'll upvote).

That said, you should really not care too much about your score and downvotes (you've probably been told this before when you lost your 100 points in the lounge back then). Sure thing, it's annoying to get downvotes (even when you get more upvotes in total), but hey... that is only natural. Funnily, this is how real perception and real intelligence work -- real intelligence is not intelligent at all, and real perception is not based on reality (not much, anyway). Both are highly selective and biased, whether we like it or not. :-)

(I'm wondering whether that is evolutionary, and it probably is -- strongly noticing and remembering a negative thing probably helps you surviving in the wilderness.)

#14 wodinoneeye   Members   -  Reputation: 618

Posted 14 October 2012 - 11:43 AM

Looking at that thread, it seems to me that you upset someone by insisting after several people who have done game AI work told you (several times) that it doesn't work that way and doesn't make sense in practice.

Though I think that even being told "dude, what you're saying is total nonsense, it really works like this..." may be useful to a reader (who might have a similar idea). Insofar, I don't see your posts as so terribly bad that they deserve a downvotes, at least not the first ones (which I'll upvote).

That said, you should really not care too much about your score and downvotes (you've probably been told this before when you lost your 100 points in the lounge back then). Sure thing, it's annoying to get downvotes (even when you get more upvotes in total), but hey... that is only natural. Funnily, this is how real perception and real intelligence work -- real intelligence is not intelligent at all, and real perception is not based on reality (not much, anyway). Both are highly selective and biased, whether we like it or not. :-)

(I'm wondering whether that is evolutionary, and it probably is -- strongly noticing and remembering a negative thing probably helps you surviving in the wilderness.)




From the start I said AI 'offline' and elaborated on how that would be probably be used in development tools long before in-game 'real' AI would be used (if ever).

But I guess some people dont bother to read what they consider a 'wall of text' (twitter generation...) and/or missed/couldnt understand what I actually said (repeatedly)..

It looked like it was mainly one person whom, since I answered with posts a half dozen times in a dialog discussion, could easily just go down the thread and downcount each one (they all had timestamps within the same minute). The other onesies/singletons I saw didnt worry me much



--

Anyway looks like they did add changes to the system that eliminated alot of the previous issues from longs ago

Metering it (15 seconds each submission?) I suppose could disuade at least some such behavior - though maybe difficult on a web based system for timings of that kind to work. A larger number of downs (than 5-6?) would be stopped by the daily down limit. The -1 penalty can be countered by irrelevant upcounts (if the person does that kind of thing alot).

Anyway is the eternal problem where you have to accept 'good enuf for most situations'

Edited by wodinoneeye, 14 October 2012 - 12:12 PM.

--------------------------------------------Ratings are Opinion, not Fact




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS