Rating Players

[hotShot]

Requirements for a rating system

A working rating system should allow you to predict the most likely result of e.g. a team game based on the rating of the players.

 

To get such a rating system you would have to have an idea how the rating of a single player effects the combined rating of a pair or team.

You need to define a minimum number of boards that have to be played before you can assume that the results are no longer disturbed by the lack of agreements or simple misunderstandings.

 

If there were such a rating system, the consequences would be:

Losing against better player would not chance your rating.

Winning against weaker player should not change your rating.

 

Why would e.g. the Lehmann rating work well in my local club, but can't work on BBO

Do you know how a pair tourney is scored with MP's?

Lets assume the top 14 world class pairs meet to play a 7 table Howell tourney. At the end of the tourney you score the result and guess what, the winner of such a tourney will have a little more than 50% while the loser will have a little less than 50%.

 

Now lets get a good intermediate pair and put in in a tourney with beginners and novices. After the tourney you score the result and you will find, that the winner will have more than 60% and the loser will perhaps have less than 30%.

 

The results of a MP tourney will be close together if all player are of about the same strength and they range will be wider if the players skill differ a lot.

 

In you local club you will always play in about the same group of people, so the field of this week is about the same as the field next week. So the results are comparable.

 

BBO has to many player, are the player who score less than 50% in the "Bermuda Bowl Participants" club really worse than those top scorer from the "Novices and Beginner" club?

 

What are the social impacts of a inefficient rating system?

In an inefficient rating system:

Pairing/Teaming up with weaker player would spoil your rating.

Playing against strong player could ruin your rating.

Playing weak player would improve your rating.

Every board played would be scored.

 

In such an environment one would need to know potential partners rating so you don't accidentally pick a weak one. One should not sub into tourneys, because you could get a weak partner and inherit bad results.

 

People would run from the table/tourney, if they are about to get a bad score.

 

The rating would be the dominant factor for getting into a decent game.

People could/would lie about their rating, if it is not displayed there would be a demand to publish the ratings.

To get the desired rating things like "bunny bashing" or cheating could come on more peoples mind.

 

Some of these social impacts might also occur if in fact a working rating system would be in place.

[babalu1997]

Pinned!!

 

I think thsi should be pinned with Fred`s recent comment concerning ratings.

[TimG]

Note that I am not referring to finding a great method of computing player ratings. This is mostly about policies about when/where/how to display playing ratings.

HotShot, I think you should assume an efficient, fair and accurate rating system and then address Fred's concerns about when, where and how to display the ratings. It sounds to me that the bigger hurdle for implementation is the social impact of a rating system rather than the ability to devise a reasonably accurate rating system.

[helene_t]

No matter how (in)accurate and (un)biased the rating is, the social impact will be related to the perceived accuracy/bias of the rating. There will always be a widespread belief that playing with good partners(or bad opps) is good for your rating, and there will at the same time be a widespread belief that is it bad for your rating. Largely unrelated to reality.

 

May sound like a bold statement but at least that is my experience from StepBridge. Their rating system is so bizarre and complicated that even experts are unable to say anything intelligent about its accuracy and biases. Nevertheless, most players are convinced that it is biased in a particular direction, typically that is is biased against them and that their rating therefore ought to be higher than it is.

[Mbodell]

There is already a relatively effective rating system for MP in use in some parts of the ACBL. See http://www.coloradospringsbridge.com/pr.htm

 

It does address the strength of field and soed address the differing strength of individuals within a partnership (and also rates established partnerships as a unit).

 

I echo the sentiments that Fred said in the other thread the issue wasn't so much coming up with the right rating system as it is knowing how to display it. I'd suggest a trial place to begin displaying it would be with the my hands web page (I.e., not at all online in the bbo client).

[mohitz]

Since, it is generally agreed that the main problem is not in finding the ideal rating system but in the concept of rating players itself.

 

Why not come up with pros and cons of having a rating system just to streamline the discussion? I will give it a try.

 

Advantages of a rating system:

1) If the rating system is accurate enough, players can find partners/opponents/teammates of same skill level and hence have a better experience of playing on BBO

2) Having a rating system will encourage players to put more effort in the game and improve(perhaps)

 

Disadvantages of a rating system:

1) Lots of people would be reluctant to have their ratings displayed.

2) Whatever rating methodology you use, i think it will always be possible for people to find holes in the system and abuse the ratings.

[hotShot]

Note that I am not referring to finding a great method of computing player ratings. This is mostly about policies about when/where/how to display playing ratings.

HotShot, I think you should assume an efficient, fair and accurate rating system and then address Fred's concerns about when, where and how to display the ratings. It sounds to me that the bigger hurdle for implementation is the social impact of a rating system rather than the ability to devise a reasonably accurate rating system.

Although I think that it must be possible to rate player, I'm sure that we are still far away from a solution.

 

Many of those who suggest a rating system, seem to think that it's simple to get some sort of ranking from myhands.

 

My point is that it is not that simple.

 

Until there is a working rating system, it makes no sense to me to think about when to display radomized numbers.

[hotShot]

No matter how (in)accurate and (un)biased the rating is, the social impact will be related to the perceived accuracy/bias of the rating. There will always be a widespread belief that playing with good partners(or bad opps) is good for your rating, and there will at the same time be a widespread belief that is it bad for your rating. Largely unrelated to reality.

...

You raise an important point here, a rating system also has to be simple enough to be understood.

But there is still a risk that "urban legends" and "conspiracy theories" will dominate the perception.

[awm]

Suppose for the moment that we had a highly efficient, highly accurate rating system. It's still not obvious how we'd want this displayed on BBO, and there would still be some undesirable social effects. For example:

 

(1) People would be reluctant to play when they are not up to their "best game" because their rating would go down. Presumably the rating system controls for strength of partner/opponents, but it's not going to control for the hands I played during the weekend I was in bed with the flu, or the hands I played to de-stress before sleep after working thirty straight hours. Thus people who care about their ratings will play less than before.

 

(2) People would discover that they are worse than they think they are. In some cases this might cause people to become discouraged with bridge in general. This might be especially bad for players who are getting older and definitely used to be better than they are now, watching their ratings decline over time.

 

(3) People will become more picky about partners. No one wants to play with a partner who is "much worse than them" (barring teacher-student type situations or friendships away from the table). This will make it harder to find pickup games, and the amount of complaining about "bad pickup partners" may actually get worse rather than better.

 

(4) However good the rating system, it's still likely that there will be an initial period when players new to BBO have not played enough hands to get an accurate rating. Other players may be reluctant to play with/against these people until their rating has stabilized, making it hard to find a game (and extremely hard to find a decent game) even if the "new to BBO" person is actually a good player. This might drive people away from BBO.

 

(5) It's always possible on BBO to create a new ID. I suspect that a lot of the people with truly bad ratings will do this. That increases the seriousness of problem 4 (since "new to BBO" rated people are often bad players creating a new persona rather than true newbies).

[TimG]

awm, I think just about all your problems can be solved by not making all play rated play. In this way people can opt into the rating system and control when they want to play rated hands. If you're sick or tired or playing with a pick-up partner or watching the game in the background, you can play in an unrated game. And, people could choose never to enter rated games and thus never be rated.

 

As far as initial break-in period, I think a good rating system would handle that without much difficulty. It would not be a simple case of assigning new players an average rating, but rather assign them a rating based upon their first x results.

 

I play a few ACBL speedballs. My motivation for playing in them rather than in the main bridge club is twofold: 1) the quality of play is generally higher, and 2) the game is not subject to players coming and going. I would happily pay my $1 to play in a tournament that was rated rather one that awarded points. A rating, and a room for rated players, could help considerably in 1) above.

 

BBO owners may read this and think: It sounds like rating you will cost us business because it will make it easier for you to find good games that don't have an entry fee. But, I would be willing to play in a certain number of rated tournaments over time to maintain my rated status. In essence, BBO would be charging me for my rating by requiring play in a certain number of tournaments to maintain a rating, but I'd be OK with that.

[awm]

While Tim is right to an extent, an awful lot of the demands for a rating system come from people who are frustrated by their inability to find a "good enough" partner in the main bridge club. A system where ratings were based on performance in particular tournaments and where it was easy to "opt-out" of the rating system entirely would not substantially address this frustration.

 

Obviously it might have other benefits.

[Jlall]

On OKB almost no one played unrated games. Generally if they did they were not very good. If you are a good player who wants to play unrated games it might be quite hard to find a good game. Even if you are not a good player it will limit your options greatly since almost everyone will be playing rated.

 

Of course BBO is now much bigger than OKB ever was, and OKB charged membership fee which kinda biases the results of how many play rated vs unrated a lot (if you're willing to pay 100 bucks a year, you'll probably want to play rated), so maybe this wouldn't be the case on BBO.

[TimG]

I think the people that can't find a "good enough" partner in the main bridge club are going to be the same people who think a rating system is not accurate (for whatever reason) in their case.

[TimG]

Justin, I think ratings should be opt-in rather than opt-out.

 

Here is what I envision.

 

Rated tournaments. Players with an established rating can play in a room for rated players (but the play there would not be rated). In order to maintain a rating, a player would have to play semi-regularly in rated tournaments.

 

The main bridge club would remain open to all, rated or not.

 

A good player could go to the main bridge club or the rated bridge club and play unrated games any time he wanted without any stigma associated with non-rated games.

[junyi_zhu]

Requirements for a rating system

A working rating system should allow you to predict the most likely result of e.g. a team game based on the rating of the players.

 

To get such a rating system you would have to have an idea how the rating of a single player effects the combined rating of a pair or team.

You need to define a minimum number of boards that have to be played before you can assume that the results are no longer disturbed by the lack of agreements or simple misunderstandings.

 

If there were such a rating system, the consequences would be:

Losing against better player would not chance your rating.

Winning against weaker player should not change your rating.

 

Why would e.g. the Lehmann rating work well in my local club, but can't work on BBO

Do you know how a pair tourney is scored with MP's?

Lets assume the top 14 world class pairs meet to play a 7 table Howell tourney. At the end of the tourney you score the result and guess what, the winner of such a tourney will have a little more than 50% while the loser will have a little less than 50%.

 

Now lets get a good intermediate pair and put in in a tourney with beginners and novices. After the tourney you score the result and you will find, that the winner will have more than 60% and the loser will perhaps have less than 30%.

 

The results of a MP tourney will be close together if all player are of about the same strength and they range will be wider if the players skill differ a lot.

 

In you local club you will always play in about the same group of people, so the field of this week is about the same as the field next week. So the results are comparable.

 

BBO has to many player, are the player who score less than 50% in the "Bermuda Bowl Participants" club really worse than those top scorer from the "Novices and Beginner" club?

 

What are the social impacts of a inefficient rating system?

In an inefficient rating system:

Pairing/Teaming up with weaker player would spoil your rating.

Playing against strong player could ruin your rating.

Playing weak player would improve your rating.

Every board played would be scored.

 

In such an environment one would need to know potential partners rating so you don't accidentally pick a weak one. One should not sub into tourneys, because you could get a weak partner and inherit bad results.

 

People would run from the table/tourney, if they are about to get a bad score.

 

The rating would be the dominant factor for getting into a decent game.

People could/would lie about their rating, if it is not displayed there would be a demand to publish the ratings.

To get the desired rating things like "bunny bashing" or cheating could come on more peoples mind.

 

Some of these social impacts might also occur if in fact a working rating system would be in place.

Well, IMO, a rating system can never be very accurate to measure an individual player's strength in bridge because bridge is a game of partnership. On the other hand, bridge rating for partnerships isn't difficult at all (Certainly, two type of rating should be implemented, IMP and MP). It's just very strange to me that few really want to implement a rating system in bridge based on pairs. Also, a rating for team is indeed the simplest for bridge cause bridge can simply borrow chess' rating system for teams(with some modifications for sure).

[Echognome]

Forget ratings!

 

A couple of alternative ideas instead:

 

A Ladder System

 

I think what might be fun would be to have a team ladder. It works simple enough. You start at a certain point (let's say for argument's sake, about 1/3 the way from the bottom). Whenever your team plays another team, the winner moves up the ladder and the loser moves down the ladder. I'm sure there are some obvious issues to solve with people moving up the ladder and then not playing, but someone should be able to come up with some solution. Maybe you have a minimum number of ladder matches you need to play to stay on the ladder. It shouldn't matter if there are ties on the ladder for those who have not played each other.

 

A Perpetual League Play

 

Have different divisions and you start off in the bottom division. You play other teams in that division over a set period of time, say a month. You must play X matches a month to stay in the league or you drop a divison (or drop to the bottom). If you have played your X matches and have Y percentage of wins (say 80%), then you move up a division. If you have played X matches and have Z percentage of wins (say 20%), then you move down a division. Teams can be comprised of more than 4 players (say 6 or 8) to ease the burden on being available for play.

 

In both cases, maybe you have a sign-up for matches in the system, akin to what you currently have for tournaments. That eases the scheduling burden as you just sign up to the allotted times. BBO can then take a small cut for these team matches (maybe $1/team) and everyone can enjoy it.

[bid_em_up]

Just out of curiosity, why does the rating need to be displayed for everyone to see? Assuming a reasonable rating system could be established, simply show it to each person when they log on so that they are the only person who actually knows what their "rating" is. Hopefully, this might convince at least some people to assess their profile "level" somewhat more accurately.

 

Other random thoughts:

 

It could be an optional item to be displayed on profile. (Check a box for on or off).

 

Certain "levels" could be prevented from claiming to be Expert or World Class status based upon their rating proficiency. This doesn't have to be based in rocket science. It's quite irritating to look at someone's MyHands records who has expert/WC in their profile, and yet, they have a negative 2.5 imp score in 1500 hands. It's fairly safe to say, this person is not Expert, much less World Class, and if that hurts their itty bitty feelings....well, so be it.

 

If somebody decides they don't wish to see how good/bad they are performing, it should be a simple matter to make it possible to "opt out" of the rating system entirely.

 

just my $0.05 (inflation is killing me)

[TimG]

Forget ratings!

 

A couple of alternative ideas instead:

 

A Ladder System

 

I think what might be fun would be to have a team ladder. It works simple enough. You start at a certain point (let's say for argument's sake, about 1/3 the way from the bottom). Whenever your team plays another team, the winner moves up the ladder and the loser moves down the ladder. I'm sure there are some obvious issues to solve with people moving up the ladder and then not playing, but someone should be able to come up with some solution. Maybe you have a minimum number of ladder matches you need to play to stay on the ladder. It shouldn't matter if there are ties on the ladder for those who have not played each other.

 

A Perpetual League Play

 

Have different divisions and you start off in the bottom division. You play other teams in that division over a set period of time, say a month. You must play X matches a month to stay in the league or you drop a divison (or drop to the bottom). If you have played your X matches and have Y percentage of wins (say 80%), then you move up a division. If you have played X matches and have Z percentage of wins (say 20%), then you move down a division. Teams can be comprised of more than 4 players (say 6 or 8) to ease the burden on being available for play.

 

In both cases, maybe you have a sign-up for matches in the system, akin to what you currently have for tournaments. That eases the scheduling burden as you just sign up to the allotted times. BBO can then take a small cut for these team matches (maybe $1/team) and everyone can enjoy it.

Build it and they will come. Let me know when you've got it started.

[matmat]

i'll just put in a plug for what I would like to see...

 

(number of hands played)/(number of partners)

[Jlall]

Justin, I think ratings should be opt-in rather than opt-out.

 

Here is what I envision.

Fair enough, that may work

[barmar]

Just out of curiosity, why does the rating need to be displayed for everyone to see? Assuming a reasonable rating system could be established, simply show it to each person when they log on so that they are the only person who actually knows what their "rating" is. Hopefully, this might convince at least some people to assess their profile "level" somewhat more accurately.

 

Other random thoughts:

 

It could be an optional item to be displayed on profile. (Check a box for on or off).

 

Certain "levels" could be prevented from claiming to be Expert or World Class status based upon their rating proficiency. This doesn't have to be based in rocket science. It's quite irritating to look at someone's MyHands records who has expert/WC in their profile, and yet, they have a negative 2.5 imp score in 1500 hands. It's fairly safe to say, this person is not Expert, much less World Class, and if that hurts their itty bitty feelings....well, so be it.

 

If somebody decides they don't wish to see how good/bad they are performing, it should be a simple matter to make it possible to "opt out" of the rating system entirely.

 

just my $0.05 (inflation is killing me)

If the rating isn't displayed, it isn't useful. The only people who are about ratings are OTHER people, who want to decide whether to partner with you, or whether they're willing to play against you. On OKbridge, table hosts typically advertise a range of Lehmans that they're willing to accept at their table (not too much lower than theirs because they want decent competition, but not too much higher because they don't want to be out of their league).

 

If displaying ratings were optional, generally people with good ratings will display them, people with bad ratings will not. Anyone who is looking for a good partner will simply ignore people with hidden ratings, on the assumption that they must be bad if they're not willing to show them.

 

What's strange, though, is that other games don't seem to have this problem. A rating system has been part of chess for many years. Do chess players with poor ratings get discouraged and stop playing, or does it spur them to keep trying to improve?

[Old York]

Self rating works reasonably well but there will always be exceptions

BBO masterpoints are awarded to players who are prepared to play/sub in certain tournaments, but many "novices" build up high numbers by playing in BiL tournies.

 

I would like to see a way of restricting tournies to masterpoint holders, and maybe have the option to set the level to 20+ MP etc.

 

Tony

[TylerE]

Self rating works reasonably well

In what parallel universe?

[suokko]

There is one big problem in rating systems. Most of people will take ratings as serius competition. That will change change attitude to a lot worse for random social games.

 

Also indivual ratings don't realy match the actual skill level of players in bridge. There is often huge difference in biding and defense performance depending on partner. If someone can play good bridge in regular partnership it doesn't tell how good random partner she would be.

 

There is relatively simple rating system for tournaments that have different level of players.

 

Take average rating of top 10%. That is competition rating ©

 

Then from result calculate average score of top 10%. That is average result (A)

 

other variables are number of competitors (N) and result ® and position in rankings (P)

 

rating points=C*(R/A)*(1-0.15*(P-1)/(N-1))

 

R and A has to be positive values so zero in IMP scoring would need to be tanslated so that everyone has positive result for rating.

 

Then actual pair/player rating is average 5 best results in last year. if less than 5 tournaments has been played rating is reduced 3% for each.

 

Rating list is published periodicaly (every month). Ratings are scaled so that the first ranked has always score of 100.

 

Unrated pairs/players join a tournament are asigned rating of 70.

 

This system is slightly modified version of the system that is used in Finnish orienteering ranking. This gives quite good rankings for different level of competitions.

 

I agree with Gnome's ladder idea if people want competitive rated games. Of course that would require improvements to the team game system. Ladder rating could degenarate 5-10% towards starter rating.

 

here is a list of features/improvements that would help:

- Preregistering to a team

- Creating team match against random team

- Some limitation to team game options for ladder match

- Selecting TD

[helene_t]

If a rating system was based on robot reward tournaments only, it would avoid two of the major pitfalls of rating systems:

- Your human partner would not be in a position to ruin your rating, so you would not need to avoid bad/overrated partners, and you would not make fights with them of the "you ruin my rating!" type that are so frequent on sites with rating.

- There would be no extra incitement to cheat (or to accuse other people of cheating) since you can't cheat with the robot award tourneys. (OK, there may be good players who would be willing to play robot award using someone else's account to boost the client's rating and getting payed for it, but I don't think that would be so widespread).

 

People could still manipulate the system by creating multiple accounts, playing some robot awards with each, and then continue with the one that got most lucky. But the money and time that would cost would deter most from doing so.