Jump to content

  • Log In with Google      Sign In   
  • Create Account


Lounge effect on your rating


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
55 replies to this topic

#1 Dave Astle   Senior Staff   -  Reputation: 2305

Posted 02 April 2006 - 01:00 PM

After reading an assertion earlier today that people who frequently post in the Lounge have higher ratings than those who do not, I asked Michalson to run a few queries on the database to see if there is any validity to that claim. What follows is the report which he provided to me, which I think some of you will find interesting. Beyond what is discussed in the report, I also looked at the data for the top 50 highest rated members, and found the following:
  • On average, 38% of their posts have been in the Lounge. In fact, if I sampled any 5 of them sequentially based on their rating, the average Lounge post percentage (LPP) only varied by a point or two
  • 10 of them have a LPP of less than 10%
  • Only 15 of them have a LPP over 50%. Only 3 of those people are in the top 20.
I then looked at the 50 lowest rated members, and found the following:
  • They have an average LPP of 65%
  • 5 of them have a LPP of less than 10%
  • 36 of them have a LPP over 50%. 17 of them were over 90% (with 7 at 100%).
Based on these observations, and the more extensive analysis described in the report, there seems to be no evidence to support the notion that frequent Lounge posting will improve your rating. If anything, the opposite seems to be true. On to the report.
The sample was created by examining posts made after the rating system was introduced. The 10,000 most active users were chosen (since they were most likely to have been seen and rated), excluding staff, moderators and the anonymous poster. The first graph shows the percentage of a user's posts in the lounge compared to their average rating. Posters on the extreme left have never seen the lounge (0% of posts in the lounge), while posters on the extreme right always post in the lounge (100% of posts in the lounge). The vertical shows the average rating of posters in a given percentile. As you can see there is a correlation between where the users post and what their rating is – those who post in the technical forums have stable ratings higher than 1000. Those who post in the lounge seem to receive both more positive and more negative ratings, resulting in a wide degree of variation, and many more extreme positive and negative ratings. Next we see the number of posts overall, compared with the user's average rating. The position (1 to 99) is the percentile of their number of posts compared to the rest (this way the data is spread out, rather then compacted to the low post count side of the chart). Users in the 99th percentile (right side) are those with thousands of posts. You can see that as the number of posts increases, the rating also increases right from the start. Next we see the same graph, but only counting posts in the lounge. Here posts in the lounge do not have any measurable effect on ratings up until the 85th percentile. At the 90th percentile, there is suddenly a rapid jump in ratings that extends about 100 points past the high reached by regular posters. The same two graphs can also been seen as pure data. First all posts, then lounge posts. Again we see that when comparing post count to rating we get a smooth curve, while the lounge post count to rating correlation is almost non-existent (though percentage seems to as seen in the very first graph).

Sponsor:

#2 boolean   Members   -  Reputation: 1702

Posted 02 April 2006 - 01:23 PM

what the heck happened to the last graph =/

#3 AnonymousPosterChild   Members   -  Reputation: 168

Posted 02 April 2006 - 01:27 PM

Quote:
Original post by boolean
what the heck happened to the last graph =/


Somebody sneezed on the monitor.
With love, AnonymousPosterChild

#4 cowsarenotevil   Crossbones+   -  Reputation: 1929

Posted 02 April 2006 - 01:28 PM

The lounge seems to be very neutral toward my rating. Most of my positive ratings seem to have come from specific threads in the technical forums, and from people who had known me prior to the introduction of the rating system.

I do get rated occasionally when I haven't posted anything helpful, but it seems to be a negative rating just as often as a positive one. And while most of my posts have indeed been in the lounge, the sheer number of things I've posted overall means that the number of posts I've made in the technical forums is still reasonably high.

EDIT: See, see! It's true! I just got rated down, and I haven't posted much of anything recently. :D

Anyway, just out of curiosity, I took my post count, subtracted all of the replies I've made in the lounge and all of the threads I've started (anywhere) to get the total number of non-lounge replies I've made. It turns out it's 3,120 (not skewed by journal posts as I very rarely if ever post in people's journals).

So despite the fact that my LPP is a little bit under 65, the sheer number of non-lounge posts strikes me as most relevant.

[Edited by - cowsarenotevil on April 2, 2006 8:28:24 PM]

#5 3uha39hr32hr89h23r32ar   Members   -  Reputation: 100

Posted 02 April 2006 - 01:29 PM

Quote:
Original post by boolean
what the heck happened to the last graph =/


He scribbled. (¬_¬)

My rating was raped via the lounge, or so I believe anyway.

#6 Boku San   Members   -  Reputation: 428

Posted 02 April 2006 - 01:53 PM

Just going to pop in for a question...

Rhino, can you do the same thing you did above (lounge effect on rating), but constricted to the Q4 2004?

Things change.

#7 kSquared   Members   -  Reputation: 1356

Posted 02 April 2006 - 01:57 PM

My proposed explanation for the so-called "Lounge effect" centers on the fact that users do not take the time to view all of a high-ranked poster's posts. That is, the proportion of page views for a user's posts in any given forum are not well-correlated with the number of posts made in that forum. Frequently-viewed forums get more page views, and so a post by a user in a popular forum is seen more often and is therefore remembered by more users than posts in less popular forums.

A given poster in the Lounge -- let's call him Bob -- is unlikely to post only in the Lounge. But unless a given GD user -- let's call her Alice -- also frequents the exact same forums as Bob, the only posts by Bob that Alice will remember are the ones Alice has seen. Since the Lounge has more posts than any other forum, it is not unreasonable to believe that it gets frequented more than any other forum. If you were to rank Bob's posts by how many times they've been seen, you would probably observe that Bob's Lounge posts are at the top of the list and his other forum posts are lower down.

In other words, (number of Bob's posts in forum X)/(number of Bob's posts) is not the same ratio as (number of views of Bob's posts in forum X)/(number of views of Bob's posts). Because the non-viewed posts cannot be remembered by users, the lion's share of a fluctuation in Bob's rating will be attributed by the community to the posts that are viewed the most.
- k2
"Choose a job you love, and you'll never have to work a day in your life." — Confucius"Logic will get you from A to B. Imagination will get you everywhere." — Albert Einstein"Money is the most egalitarian force in society. It confers power on whoever holds it." — Roger Starr{General Programming Forum FAQ} | {Blog/Journal} | {e-mail me} | {excellent webhosting}

#8 Cold_Steel   Members   -  Reputation: 835

Posted 02 April 2006 - 01:58 PM

Your chart proves nothing:


See, charts can say anything you want them to [wink]

#9 Michalson   Members   -  Reputation: 1657

Posted 02 April 2006 - 02:24 PM

Quote:
Original post by boolean
what the heck happened to the last graph =/


It's a distribution graph, only with the points removed (there are so many points that it's just a big blob) so just the directionals remaining.

The posts vs rating graph, first one, is relatively smooth (except for the extreme post count area, where there are not enough users to provide a strong sampling), showing a correlation between increased user activity and higher ratings. If you've posted 3 times, don't expect to have a rating of 1500, no matter how good your posts where. But if you've posted 1000 times, and are generally a good poster, you'll probably get a lot of rate ups.

The lounge posts vs rating graph, second one, is a different story. As you can easily see, it's all over the place. That's because there is no correlation between the number of posts you have in the lounge and your rating. Only a very tiny one can be seen, and that's the result of a few oddball super-data points like nes8bit, and the simple fact that a lot of people with a decent number of lounge posts also have a descent number of posts overall, which means you're really seeing an after image of the first graphs post to rating correlation, buried inside the sea of random statistical noise that is the lounge posts vs rating non-correlation.

#10 MrEvil   Members   -  Reputation: 970

Posted 02 April 2006 - 02:30 PM

Not sure if everyone knows this so...

If you want to find out your LPP, go to your profile, click "Show all replies by this user", in the "Forum" combo box choose "GDNet Lounge", and Update. It will tell you how many results were returned, which you can use to calculate your percentage.

Er, wait, just realised, this doesn't include journal posts, so it's not really all that accurate. Mine's about 60-70%, I used to frequent the technical forums a lot more [sad]

#11 xanin   Members   -  Reputation: 433

Posted 02 April 2006 - 02:51 PM

Quote:
Original post by MrEvil
Not sure if everyone knows this so...

If you want to find out your LPP, go to your profile, click "Show all replies by this user", in the "Forum" combo box choose "GDNet Lounge", and Update. It will tell you how many results were returned, which you can use to calculate your percentage.

Er, wait, just realised, this doesn't include journal posts, so it's not really all that accurate. Mine's about 60-70%, I used to frequent the technical forums a lot more [sad]


Ugh, thanks alot. Turns out I'm a 74% lounge poster. I gotta get out of this place.

#12 Nytegard   Members   -  Reputation: 820

Posted 02 April 2006 - 03:10 PM

I guess I'll take the devils advocate on this.

Whereas when I was in college, the curve was always based with the extreme points taken off ( the few who always scored above the norm, and the few who always scored below). Granted, I don't deny that the majority of the top 50 are elsewhere, answering real questions.

kSquared mentioned that the rating is based on the most viewed. Whereas the lounge is the most viewed (as per number of posts), I seriously doubt that everyone who comes to gamedev lives their lives in the lounge. Let's be realistic. If your goal was to just chat about junk all day, there probably are better places than gamedev that share your interests.

At the same time though, I doubt trolls are going to be visiting the techincal forums that often. The moderators are pretty good about deleting, moving, or locking posts that don't belong. The majority of posts I've seen which have rated users down have either been about ratings, a troll, a subjective opinion (politics, religion, language comparisons, etc), etc. And they can quickly shred a users ratings. The techincal forums tend to be more objectively based questions.

When I've seen posts with a wrong answer such as "How do you convert a string to a double", and someone gives a wrong answer saying to use "atoi", usually another post is made correcting the previous user with no ratings effect, not a "Shut up noob" post with a low rating. I have yet to see a flamewar go on about how to initialize Direct3D.

I guess it comes down to that I personally feel the lounge effect has a huge effect on the people not 1500+, but rather the people in the 1000-1300 range.

I just have a few questions if anyone who has information is willing to devulge this information.

How many unique visitors are there to the techincal forums vs the lounge? (more unique visitors to the tech forums would explain a higher rating overall to the people who answer tech questsions, and this is what I think is happening).

How many overall positive ratings are given in the lounge?

How many negative ratings are given in the tech forums? (I can't even believe that it would be possible to be rated down 1000 pts in the tech forums legitimately for giving wrong answers.)

And for the people rated in the 1000-1300 range, what percent of those are lounge based versus technical forum based?

If I have a technical question, and I receive an answer, I'll most likely know in an hour or two max if what they're telling me is right, regardless of if they had a 0 rating or 100000 rating. And if what they stated works, even if they had a 0 rating, it works, and no one has the rights to state that they're an idiot on that question because of their thoughts on their unpopular beliefs they made in the lounge.

#13 Servant of the Lord   Crossbones+   -  Reputation: 16685

Posted 02 April 2006 - 03:21 PM

As proof of this topic; I posted 535 (or 536, now) posts in the lounge out of
816 total. More than three-quarters of my posts have been in the lounge. Now look
at my rating...
And actually, as recently I've posted in the lounge less, my rating has been up about fifty-ish points. And then down twenty as I made a few more lounge posts.
As my 536th lounge post, I am certain I will loose a few points for this post also.

Thanks for the data Myopic Rhino! (And this very site)

It's perfectly fine to abbreviate my username to 'Servant' rather than copy+pasting it all the time.

[Fly with me on Twitter] [Google+] [My broken website]

All glory be to the Man at the right hand... On David's throne the King will reign, and the Government will rest upon His shoulders. All the earth will see the salvation of God.                                                                                                                                                       [Need free cloud storage? I personally like DropBox]

Of Stranger Flames - [indie turn-based rpg set in a para-historical French colony] | Indie RPG development journal


#14 nilkn   Members   -  Reputation: 960

Posted 02 April 2006 - 03:34 PM

Quote:
Original post by MrEvil
Not sure if everyone knows this so...

If you want to find out your LPP, go to your profile, click "Show all replies by this user", in the "Forum" combo box choose "GDNet Lounge", and Update. It will tell you how many results were returned, which you can use to calculate your percentage.

Er, wait, just realised, this doesn't include journal posts, so it's not really all that accurate. Mine's about 60-70%, I used to frequent the technical forums a lot more [sad]


I'm surprised. Using this method, I have an LPP of 25%. Apparently I used to be helpful. [lol]

#15 Cypher19   Members   -  Reputation: 768

Posted 02 April 2006 - 04:04 PM

Is this report why we had no GDNet forums for half the day?

#16 Oberon_Command   Crossbones+   -  Reputation: 1782

Posted 02 April 2006 - 04:13 PM

About 46% LPP for me, the rest of my posts are in the technical forums. Looks like I'm fairly typical, I guess, since my rating is fairly stable.

#17 capn_midnight   Crossbones+   -  Reputation: 1375

Posted 02 April 2006 - 04:20 PM

88% LLP and going strong at ~1400. Heck, I've even tanked from 1400 to 1200 on occasion. I think it's mostly the 3H-GDC threads and the limited posting I do in the .NET forum.
[See some of My Projects] - [Find me on twitter tumblr G+ Github]

#18 Oluseyi   Staff Emeritus   -  Reputation: 1670

Posted 02 April 2006 - 04:21 PM

Quote:
Original post by Cypher19
Is this report why we had no GDNet forums for half the day?

No.

#19 owl   Banned   -  Reputation: 364

Posted 02 April 2006 - 04:29 PM

Posting in the lounge makes you be rated by no parameter other than peoples feelings. You may feel you're posting the most insightful thought and yet be rated down, cause, naturally, that may fire lots of differents feelins on people.

#20 capn_midnight   Crossbones+   -  Reputation: 1375

Posted 02 April 2006 - 04:33 PM

Quote:
Original post by owl
Posting in the lounge makes you be rated by no parameter other than peoples feelings. You may feel you're posting the most insightful thought and yet be rated down, cause, naturally, that may fire lots of differents feelins on people.


I'd say it's more related to how you post, and not what you post. Say "OMG, all dem Iraqis r terr'sts and should effing die" a pretty decent way to get rated to the pits of hell, even by people that share feelings of distrust against middle-eastern peoples.

(disclaimer: the supplied example is not an expression of the views and/or beliefs of the GDNet Staff and capn_midnight)
[See some of My Projects] - [Find me on twitter tumblr G+ Github]




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS