More dev

Published July 08, 2008
Advertisement
OK, post icons - on threads - are safe for now. I've left them off individual posts, though they might go back on; I can see them sometimes being useful to communicate the overall tone of a post (I often used to use the roll-eyes smiley when being sarcastic). We'll see. Certainly where icons are kept I think we will roll out more of them.

Design work continues... most recently I've written in the stuff about subscriptions. Paypal will still be supported, and eventually I want to look into the possibility of supporting transactions through other means, maybe such as Google Checkout.

There are still a few things on my to-do list - some smaller than others. The one that I guess is amongst the most contentious is the rating system.

It's been established that the site will be getting the ability to tag a user with one or more keywords. If you think that a particular person is "all about" graphics, or neural networks, or whatever, you can tag them accordingly. Then, when you're searching for information on a particular topic, the site will be able to point you at people who are heavily involved in that topic - the idea is that they will be the 'experts in the field.' The search will also be able to do things like identifying threads or articles that have involvement from those experts (handy if you're looking for answers), versus things that do not (handy if you're looking for questions).

The tagging system won't just be limited to technical topics. If somebody's just a really nice guy, you can tag them with 'nice guy' (or just 'nice'). If they're good at explaining things, you can tag them with 'good teacher' or 'good at explaining.' If they're impatient and ungrateful, you can tag them with 'unpatient' and 'ungrateful.'

The question is, is that enough to be useful?

Remember that the site has no concept of what a tag means. It has no inherent distinction between 'idiot' and 'guru' - they're both just words. As such it's difficult for the site to 'take action' against people who are being rude and abusive; it doesn't know which tags indicate that.

Furthermore, I'm not sure how many people will be comfortable tagging somebody as an idiot. It is, perhaps, a bit too negative, a bit too damning, and it lacks eloquence - 'idiot' isn't very descriptive. The system won't prevent it for those people who /are/ comfortable with it, naturally, but I fear that it may simply not be used by people who just want to express a vague feeling of displeasure with a person.

Add to this another oft-cited issue with the existing rating system - that people too often don't know or understand what they've been rated up or down for. We've said in the past that ratings should be awarded based on a holistic consideration of a person's contribution - that you shouldn't rate somebody without looking at their profile and seeing their other posts. Maybe they're just having a bad day. After observing the system for several years, I don't think people do this - so maybe it's worth abandoning the approach.

What I'm considering is a variation on a system I've seen at some other forums - specifically I'm thinking of TCE, though I'm sure it's elsewhere as well. Simply put, on every post, there'd be a "thanks!" and "no thanks!" button. You press the former if you want to thank a user for their contribution; you press the latter if you feel the opposite. The total number of 'thanks' and 'no thanks' are weighed up and used to calculate a karma rating for the post. A user's total karma rating is then calculated as a function of the karma ratings of all their posts.

To be clear, unlike these other systems, it would still be anonymous. You would not see who has thanked/blamed somebody for a given post, only the number of people who have done so. I'm thinking as well that it would be displayed on a colour scale rather than a numeric value, so that people don't go apeshit over tiny changes in value.

What do you think?
Previous Entry More design work
Next Entry And more
0 likes 16 comments

Comments

speciesUnknown
Quote:Original post by superpig
OK, post icons - on threads - are safe for now. I've left them off individual posts, though they might go back on; I can see them sometimes being useful to communicate the overall tone of a post (I often used to use the roll-eyes smiley when being sarcastic). We'll see. Certainly where icons are kept I think we will roll out more of them.

My thoughts on this are that the per-post icons dont make a whole lot of difference, but you can use them as emoticons. Some do, some don't.
Quote:
Design work continues... most recently I've written in the stuff about subscriptions. Paypal will still be supported, and eventually I want to look into the possibility of supporting transactions through other means, maybe such as Google Checkout.

Keeping paypal is good, thats how I pay. I personally have no need for any additional pay methods, especially when its once per year.
Quote:
There are still a few things on my to-do list - some smaller than others. The one that I guess is amongst the most contentious is the rating system.

Making any form of change will be very resisted, im not sure its actually necessary. Everybody suggests their own personal idea of what the rating should do and you will not please everybody.
Quote:
It's been established that the site will be getting the ability to tag a user with one or more keywords. If you think that a particular person is "all about" graphics, or neural networks, or whatever, you can tag them accordingly. Then, when you're searching for information on a particular topic, the site will be able to point you at people who are heavily involved in that topic - the idea is that they will be the 'experts in the field.' The search will also be able to do things like identifying threads or articles that have involvement from those experts (handy if you're looking for answers), versus things that do not (handy if you're looking for questions).

Is the tag you set visible to anybody else? if it is then this may be a mistake... e.g. when you get tagged "Gun toting maniac" or "Bleeding heart liberal pussy" because you posted in the recent Gun control thread.
Quote:

The tagging system won't just be limited to technical topics. If somebody's just a really nice guy, you can tag them with 'nice guy' (or just 'nice'). If they're good at explaining things, you can tag them with 'good teacher' or 'good at explaining.' If they're impatient and ungrateful, you can tag them with 'unpatient' and 'ungrateful.'

Does this mean it will be a selection from an explicit list, or does it take any string?
Quote:

The question is, is that enough to be useful?

Not really, as people show different personalities in different forums, but I for one do not want to get contacted by the last 4 people I helped with vector math or SDL because they remember me from last time.
Quote:
Remember that the site has no concept of what a tag means. It has no inherent distinction between 'idiot' and 'guru' - they're both just words. As such it's difficult for the site to 'take action' against people who are being rude and abusive; it doesn't know which tags indicate that.

Furthermore, I'm not sure how many people will be comfortable tagging somebody as an idiot. It is, perhaps, a bit too negative, a bit too damning, and it lacks eloquence - 'idiot' isn't very descriptive. The system won't prevent it for those people who /are/ comfortable with it, naturally, but I fear that it may simply not be used by people who just want to express a vague feeling of displeasure with a person.

linxor I would be using this definition of idiot if I had the option to label a person as such, however many people consider an "idiot" to be a person who does not agree with them or their political belieefs, or their "one true operating system" or whatever it means at the time. Its too broad and for that reason is unsuitable for an explicit list of tags.
Quote:
Add to this another oft-cited issue with the existing rating system - that people too often don't know or understand what they've been rated up or down for. We've said in the past that ratings should be awarded based on a holistic consideration of a person's contribution - that you shouldn't rate somebody without looking at their profile and seeing their other posts. Maybe they're just having a bad day. After observing the system for several years, I don't think people do this - so maybe it's worth abandoning the approach.

a simple per-post YES/NO would suffice here, but the question remains what happens to people with an already high or low rating, how will you translate the numeric value into something meaningful in the new system? If you can pull this off without making people with 1000 <= rating <= 1100 look like fools I will be happy.

Quote:
What I'm considering is a variation on a system I've seen at some other forums - specifically I'm thinking of TCE, though I'm sure it's elsewhere as well. Simply put, on every post, there'd be a "thanks!" and "no thanks!" button. You press the former if you want to thank a user for their contribution; you press the latter if you feel the opposite. The total number of 'thanks' and 'no thanks' are weighed up and used to calculate a karma rating for the post. A user's total karma rating is then calculated as a function of the karma ratings of all their posts.

To be clear, unlike these other systems, it would still be anonymous. You would not see who has thanked/blamed somebody for a given post, only the number of people who have done so. I'm thinking as well that it would be displayed on a colour scale rather than a numeric value, so that people don't go apeshit
July 08, 2008 12:11 PM
Daerax
Can i tag someone as a bleeding idiot who 'knows nothing about the importance of the iraq war'?

Can the tag not also be accompanied by weight? thus the more a person has a certain tag the higher they will be in that category. Then the tag must have a certain weight before it becomes an attribute of the user. If n number of people took the time to label you idiot then chances are that the title may be fitting.

Can you tag tags? For example what if there is a Graphics Programming Master, Graphics Programming Wizard, Graphics Guru. How will the search take this into account.

I think it might be ideal to restrict the set of available tag to a certain sized subset. Thus you can tag but can only pick from a certain set. This will eliminate redundancy, simplify search and matching logic and idiotic labelings. Ideally it would also be trivially easy to add a new item to the set of all GDNet Tags.
July 08, 2008 12:14 PM
superpig
Quote:Original post by Daerax
Can i tag someone as a bleeding idiot who 'knows nothing about the importance of the iraq war'?
Yes, if you want.

Quote:Can the tag not also be accompanied by weight? thus the more a person has a certain tag the higher they will be in that category. Then the tag must have a certain weight before it becomes an attribute of the user. If n number of people took the time to label you idiot then chances are that the title may be fitting.
Yes. It's also likely, I think, that you'll be able to give a tag a negative weighting, so if somebody is tagged as 'idiot' and you disagree then you can anti-tag them.

Quote:Can you tag tags? For example what if there is a Graphics Programming Master, Graphics Programming Wizard, Graphics Guru. How will the search take this into account.
The intention is that people only tag with 'graphics' or 'graphics programming.' The fact that you applied the tag, with full weight, is supposed to imply master/wizard/guru status.

Quote:I think it might be ideal to restrict the set of available tag to a certain sized subset. Thus you can tag but can only pick from a certain set. This will eliminate redundancy, simplify search and matching logic and idiotic labelings. Ideally it would also be trivially easy to add a new item to the set of all GDNet Tags.
I've explicitly decided I don't want to do this, but we will be employing tech to 'suggest' tags from amongst those that have already been used. Clicking on a suggested tag is easier than typing a new one, so tags should gravitate towards those that have already been used.

July 08, 2008 12:25 PM
Oluseyi


Simply clicking on the "+" or "-" applies a positive/negative rating to the post. Clicking on the individual labels that pop up when you hover over the "+" and "-" increments/decrements those specific values. A user's rating over another user is therefore composed of four parts:

Helpfulness: How helpful the rating user considers the rated user
Friendliness: How friendly the rating user considers the rated user
Humorousness: How funny the rating user considers the rated user
Aggregate: The weighted average of the rating user's opinions of the rated user.

Clicking the plus or minus signs on a post bump the aggregate rating of the user.

Thoughts?
July 08, 2008 12:28 PM
ukdeveloper
I've always felt the rating system was a bit pants, it's used for bragging rights and doesn't prove anything at all. You can get rated up for saying something funny, or you can get rated up for solving someone's problem; it's not a useful metric to go by and says nothing about someone's skills or knowledge at all. It feels like people rate you down just because they can, and it often feels like they completely disagree with you and are twisting the knife for the sake of it.

I've always been in favour of removing it entirely, but failing that I have another idea:

vBulletin has a reputation system which attaches reputation to individual posts. You are told in your control panel which user left you positive/neutral (which seems pointless)/negative rep, for which post it was and, more importantly, why it was. The rep settings can be adjusted in the admin CP and you can tweak settings like only neutral rep from new members etc.



Green is positive, you get red for negative and grey for neutral.

I think it works well, problem is people have rep wars and people vote each other up and down as favours etc. That said, if I were building a forum system from the ground up and wanted to include a feature like that, vBulletin's version is probably the one I'd model it on.

I also agree with SpeciesUnknown in that you'll find yourself faced with a hard time trying to please everyone and the rating system has been a touchy subject at least in all the three and a half years I've been here.

As for tags and post icons, I don't have anything to say on that subject for now.
July 08, 2008 12:47 PM
Mike.Popoloski
I, for one, like the rating system as it is now. The only issue I see with it is that the lounge influences ratings in ways that are completely unrelated to technical competence.

I'm with Oluseyi's suggestion, except perhaps keep it even simpler. The rating system as it stands now works because it's simple and anonymous. It says nothing more than you make of it. My only suggestion would be to split it into two parts: a rating based upon your technical competence / helpfulness, and a rating for friendliness / whatever (ie. the lounge).

I think keeping the rating as a hard and fast number is a good idea, because you want users to think about what they are going to post and the reaction it will receive, and ultimately what it will do to their rating.

Despite what some people say now, the rating system is still a good indicator for judging the usefulness of a person's information. There's the argument that a user can be rated highly for, say, C++ knowledge, but not be reliable for questions regarding OpenGL. While this is true in theory, in practice people who are competent enough to receive a high rating will A) know their own limitations better and B) be skilled across a wide range of topics.
July 08, 2008 01:00 PM
ukdeveloper
Actually, I'd be happy with the rating system in its current form if your rating is hidden and known only to you and staff/moderators instead of being shoved publicly under every single post. It would stop the bragging rights and stop people from caring so much about it along with the rating threads (which should be banned outright in the site rules IMO) I for one don't really like it when I post something I believe in and somebody disagrees, rating me down just because they can with little regards to "helpful" or "friendly" which is what the descriptions say when you make your choice on how to rate someone. It's not constructive or helpful in my mind.

I know the system will never be changed, however.
July 08, 2008 02:24 PM
_the_phantom_
The problem I can see with being able to tag people as good in certain areas is that of PM spam.

I've had a few PMs in my time as the OGL forum mod from people asking for help directly, without going via the forums, which I consider annoying at best as I have to take the time to see what the message is about and generally reply with a 'no, post in the forum'.

At least however that was limited to one topic; if people are publically known as 'good' at certain topics you are going to get a situation where people are going to start hunting them down for help directly. A situation made worse when someone who has used a piece of technology for years switches or drops it and simply doesn't want to answer questions on it any more.

I can see the tent of such a system however I also have very little faith in the human race as a whole tends to make me think this won't work all that well.

At the end of the day the rating system, love it or hate it, in it's current form has cut down on the ammount of crap which gets posted; if the new system can continue to act in that regard then all will be well in the world.
July 08, 2008 02:51 PM
Oluseyi
Btw, I see no value in post/thread icons. Give everyone an avatar, expand the smiley set and call it a day.
July 08, 2008 04:02 PM
Emmanuel Deloget
Quote:Original post by ukdeveloper
Actually, I'd be happy with the rating system in its current form if your rating is hidden and known only to you and staff/moderators instead of being shoved publicly under every single post. It would stop the bragging rights and stop people from caring so much about it along with the rating threads (which should be banned outright in the site rules IMO) I for one don't really like it when I post something I believe in and somebody disagrees, rating me down just because they can with little regards to "helpful" or "friendly" which is what the descriptions say when you make your choice on how to rate someone. It's not constructive or helpful in my mind.

I know the system will never be changed, however.

Well, if ratings are not public, why should we have rating at all? Hiding them is just like not having them. We (staff and mods) don't care about the rating of any particular individual, because - as you - we know the limitation of the system. We try to help every people, including those who have a 0 rating.

One of the major point of the current rating system is self-moderation and noise-to-signal ratio reduction. It prevent semi-stupid people to constantly misbehave on the site, and raise the general forum quality (because people try to help others in order to improve their rating). I even suspect that Oluseyi's behavior changed because of the rating system (we were told that it's not the case, but you can't imagine how he feels about his own rating... [smile]). Ok, that was a joke.

Of course, that doesn't mean the system is perfect. There are many ways to improve it, but we are well aware that any change in this area will be resisted - because we can't please everybody. At least, the current system is quite democratic - consider it as a voting system.

Regarding individual who misuse the rating system, well, what can I say? I don't think the rating system is neither misused or abused. People just use it the way they want. It becomes part of the communication, the way by which you can express your own feelings on a technical forum.
July 09, 2008 02:28 AM
jollyjeffers
Quote:The problem I can see with being able to tag people as good in certain areas is that of PM spam.

I've had a few PMs in my time as the OGL forum mod from people asking for help directly, without going via the forums, which I consider annoying at best as I have to take the time to see what the message is about and generally reply with a 'no, post in the forum'.
This would be a big thing for me as well. I don't get much direct-to-PM questioning, but it is something I'd rather discourage than encourage...


Also, tagging users sounds good to me - I'd find that much more useful than some arbitrary number! That said, I think tags should have an expiry to them - otherwise a multi-year member might carry around some "grumby b*rd" tag just because they had an off week 18 months ago. Relevance of a user-tag should diminish after, say, 6 months or so.


Jack
July 09, 2008 04:59 AM
Endar
The problem with such a broad system as this is that the way to get rid of the tags needs to go hand in hand and be almost as easy as adding tags. But the only way that I can think of getting rid of tags would be the user to lodge a protest of some sort against one or more tags that someone has given them, but then a member of staff or a mod would need to go through all of these and delete the undeserved ones, or at least the stupid ones, like "idiot" or "hopelessly republican", etc, etc.

Having a mod or a member of staff go through people's tags doesn't seem to be a workable system. If you were to give each member the ability to remove tags that other people have given them, the tags system effectively becomes useless and unable to be trusted.

Perhaps a better system would be to have a broad list of items that can be added to by mods/staff of tags to give someone. Also, like Daerax says, there should be a minimum number of people that have to give you that tag before it goes into effect and becomes visible to people and to tag searches. This would at least somewhat eliminate impulse tagging based on a single post and even out a person's set of tags and let them be more trusted (in the general case).

This is starting to sound more like the new rating system that was stickied for a while in the CSI forum about the icons that would describe each person. The only difference would be the possibility of people creating whatever tags they want. The icons could be more visible and useful in quickly seeing what other expertise a person has.

I think that a single colour representing a user's rating would be a good idea, cutting down the amount of knee-jerk complaints about being rated down, while still giving a good overall view of a person. But I also think that tagging as well as rating would end up being a problem. Having to do 2 different, seperate things to say that someone is good it doesn't make much sense. It needs to be as easy as possible.

Unfortunately I don't have much of a better idea (when I do, I shall make it known), but in the end I'm with phantom in saying that as long as the rating system makes sense and it actually works, then it is a good system.

Edit:
Quote:Original post by Oluseyi
Btw, I see no value in post/thread icons. Give everyone an avatar, expand the smiley set and call it a day.

Yeah.
July 09, 2008 05:08 AM
coldacid
Quote:Original post by superpig
OK, post icons - on threads - are safe for now. I've left them off individual posts, though they might go back on; I can see them sometimes being useful to communicate the overall tone of a post (I often used to use the roll-eyes smiley when being sarcastic). We'll see. Certainly where icons are kept I think we will roll out more of them.

I make use of post icons for replies, and I notice that others do too, generally to set the tone for their post. I'd like to see them stay around.
July 11, 2008 07:37 PM
MrEvil
I like some of the post icons – especially , and are useful.
July 11, 2008 10:20 PM
TheGilb
About the ratings system ...

As things stand in the current design, the ratings system feels 'bolted on', with very little actual use within the site. I think the rating system is a great idea, but it could possibly be put to a more effective use.

Another issue, - possibly linked - the same topics in the forums come up over and over again, and on the 'active topics' section of the site, topics that have been answered several times over already keep getting comments, and ultimately I think this leads to beginners often getting confused. Posters only need one answer.

Also the tagging system doesn't mesh with user profiles too well. Areas of expertise can be roughly divided up into the major game programming disciplines - like they are in the forums already. Graphics, Networking, Maths .. etc. If you give the user an inch to abuse the system - he will. Don't make me 'tag' you over this.

The way I see things, you could maybe borrow a few ideas from the likes of experts-exchange.com. Your rating is linked to your profile in the form of points which can be spent to ask a question, and you can earn points by answering questions. If a user accepts an answer the points are awarded from the posters account, a grade awarded, and the topic closed. Users can easily search past topics, with the best answers highlighted by their rating. Now you're building up a knowledge base.

Stats are visible in realtime on the main page, grouped by category, in the 'Hall of fame'. Now you're making a game where the users are actively encouraged to participate and share whatever knowledge they have, even if their rating is super massive. Suggest giving ratings an upper limit of a 32 bit unsigned integer. Users could also be awarded a rank like Beginner, Expert, Guru, Genius, Jedi .. whatever - and that rating could be linked to the points in their profile. Rating icons could be displayed in posts, and help guide posters better towards a good answer.

Because many gamedev participants like to spam the forums with rubbish, some forums could be flagged for 'free chat' or whatever, where the points system is completely bypassed. Now users keep spam out of the main categories where people are looking for answers, it means less work for the moderators.

Now you have a reason for ratings, you are discouraging duplicate topics already answered, you are discouraging forum spam and encouraging community participation.
July 12, 2008 07:19 AM
TheUnbeliever
This is maybe a little contrived, but here's a couple ideas somewhat related to the above poster's suggestion: what if a user could tag a post as a question, an attempted answer (associated with (a) specific question(s)?) or neither (the 'neither' category being for general posts or queries for more information (e.g. "What compiler are you using?") and responses to such).

Attempted answers could then be rated on a 4-point scale: 'Unhelpful' (Bad faith), 'good faith but technically inaccurate', 'helpful but with caveats (variable technical accuracy or helpful response mixed with abuse)', and 'model answer'. These scores for a given user's attempted answers could then be somehow aggregated (with weight of scores falling off with time - users can improve or degenerate, obviously) to provide a kind of 'rating' for a user.
July 14, 2008 09:31 AM
You must log in to join the conversation.
Don't have a GameDev.net account? Sign up!
Profile
Author
Advertisement
Advertisement