ELO system for bbo?

[lmhk]

I enjoy playing bridge as much as the next person. Much more, I enjoy playing competitive bridge. I know there are tournaments and masterpoints and even BBO prime on the site now. Instead of these platforms, how about creating an ELO system where each player is rated? I don't know how the programming would work, but rating systems exist in many games, including chess, GO, sports, esports and board games. From my experience in playing other games, having a rating system maximizes the competitiveness and fun. I also think it might be an interesting way to popularize the game (especially among younger people) and increase traffic. Currently, the only rating on BBO is a self-assessment of skill level, which by the way, is highly inaccurate (extremely incompetent "advanced" players all over the site). Of course, you can still divide the site into two sections - casual and competitive, and those who merely want to relax and play some evening bridge without having to worry about others judging your rating could just stay in the casual section.

[pilowsky]

Hi, in the short time that I've been looking at this Forum, this question has come up many times (including from me as ex-junior chess player).

Bridge is similar to all other equilibrium games such as Chess and Go, but the big dissimilarity is the requirement for people to play as a pair.

This means that if you want a rating system, there would need to be a rating system for pairs rather than individuals.

Unfortunately, many Bridge partnerships are as evanescent as steam from a boiling kettle (I could meander more through that metaphor but you get the idea).

 

There is a workaround that I have used. You can download an ELO calculator from the web and then play in regular massive daylongs where the number of competitors is often greater than 1000 every day. The quality is very mixed so it can be characterised as international tournament every day.

Map your results from these robot daylongs, and you will gain a rough idea of your equivalency to Chess ratings.

 

Clearly, stars, masterpoints and success in previous tournaments - all of which contribute to overall ranking in Bridge - are of little value in determining a player's instantaneous skill level.

This is because masterpoints accumulate like barnacles and exist to make money for Bridge organisations who then provide the tournaments that we all enjoy.

 

There is an equivalent problem in Chess in that titles such as International Master and Grandmaster are awarded based on achieving excellent results in high-quality tournaments.

This is why I suggest using large daylongs if you really want to map Elo (his name was Arpad Elo by the way) rankings onto Bridge players.

 

Of course, there is the little problem that not everyone regards robot Bridge with the same affection as I do (litotes), but them's the breaks.

[paulg]

One of the forerunners of BBO, OKBridge, had a rating system (Lehmans) but it generated behaviour that was generally seen as unacceptable.

 

A lot of members focused on their rating and would not play with weaker partners or opponents: even though the rating system protected them from weaker players, very few believed this and even fewer were willing to take the risk. This caused a lot of bad feeling when players were booted by the table host shortly after sitting down.

 

I believe that BBO deliberated avoided a ratings system initially to avoid this behaviour.

 

It is interesting to see that the English Bridge Union, and perhaps others, now have national rating schemes and the players are more accepting. Perhaps because there are more events targeted at them.

[lmhk]

This is similar to ranking systems in team-based games. Personally I had some experience gaming and I understand the frustration there would be if you feel your partner is bringing you down. It is the same as losing a basketball game because of bad teammates, even if you are the best player on the team. However, similar to other games that need teammates, I believe in the long run, good players will have an increase in rating and lesser players the opposite. Or there could be partnership ratings as well as individual ratings. As for the poor behaviour, I assume there would be some terms to agree to before participating in the rating system. I believe there is a way around this if they really wanted to do it, and personally I think it would make bridge so much more fun and competitive.

[lmhk]

I am not trying to find out what my rating is, I am merely suggesting that having a rating system could make the game a lot more fun and competitive. There are ranking systems out there in the gaming industry that are also team-based. Everyone would still get an individual rating. The idea is, if you are a good player, in the long run, given random partners, you would yield winning results and move up and vice versa. I think it is definitely plausible and more fun for everyone if they can get matched up with players of similar skill-levels.

[hrothgar]

I think it is definitely plausible and more fun for everyone if they can get matched up with players of similar skill-levels.

 

The ride isn't worth the cost of the ticket

 

Introducing rating systems introduces an enormous range of problems.

 

The most direct are a series of incredibly painful social interactions when people are either

 

A. Trying to protect their ratings

B. Trying to blame other people because they just damaged their ratings

 

Running a close second is a series of incredibly painful interactions trying to explaining to people that the rating systems says that are bad bridge players.

 

And of course, there is the never ending joy of trying to explain "How the ratings system works" and "No, the ratings system is not broken" to people who are mathematically illiterate and technophobes.

 

As was already discussed, Fred made a very conscious decision not to implement a ratings system after seeing how destructive this was to the OKB playing environment.

 

I think that he made the right choice at the time. Right now, I think that the arguments against creating a ratings system are somewhat less strong (I think that there are good enough machine learning algorithms to do a good job at creating a ratings system. 18 years back, I'm less sure). However, the critical issue is still the impacts on the social dynamics of the site. And here, I think that ratings systems are still rank poison.

[pescetom]

Right now, I think that the arguments against creating a ratings system are somewhat less strong (I think that there are good enough machine learning algorithms to do a good job at creating a ratings system. 18 years back, I'm less sure). However, the critical issue is still the impacts on the social dynamics of the site. And here, I think that ratings systems are still rank poison.

Not sure we even need machine learning. As previously discussed, the EBU NGS ranking scheme seems to work and is designed to see beyond pairs. It could be adapted to the more mainstream BBO tournaments without problems I would expect.

 

As for the social issues, you may be right. I have played on another online card game site where the rating system poisons all social interactions, for the reasons you mention. But I would be curious to know if there are counter-experiences. Does Elo create chronic social problems in online chess, which is considerably more popular than bridge? If not, could it be because the rating is accepted as site-independent and true?

[hrothgar]

Not sure we even need machine learning. As previously discussed, the EBU NGS ranking scheme seems to work and is designed to see beyond pairs. It could be adapted to the more mainstream BBO tournaments without problems I would expect.

 

"seems to work" is mighty thin soup

 

I can point out any number of flaws with the NGS system

 

If you are going to do this, do it right.

[pescetom]

"seems to work" is mighty thin soup

You said that before, but I don't remember you pointing out any actual flaws.

Have you done so to the NGS Working Group, or on BridgeWinners which would be a better place than here?

[hrothgar]

You said that before, but I don't remember you pointing out any actual flaws.

Have you done so to the NGS Working Group, or on BridgeWinners which would be a better place than here?

 

Yes I have.

They don't care.

They are happy with their archaic little scheme and aren't interested in considering other approaches.

 

In terms of the flaw, at the most basic the NGS scheme calculated results on a session by session basis. It is unable to adjust for the way in which differences in individual boards impact your results. Some board are (naturally) going to be flat. Other have a lot more room for player skill to impact results. If you're unlucky enough to play flat boards versus weak pairs and complicated boards against strong pairs you're going to have a crappy session.

 

NGS, ELO and the like are based on methods that are 40+ years old.

 

In the world of machine learning and AI, 5 years is a lifetime.

 

The EBU's attitude seems to be

 

We came up with the following.

We think it's good enough

There's no reason for us to improve anything

We aren't even interested in doing to bake off to evaluate other schemes or compare accuracy

[nige1]

Not sure we even need machine learning. As previously discussed, the EBU NGS ranking scheme seems to work and is designed to see beyond pairs. It could be adapted to the more mainstream BBO tournaments without problems I would expect.

 

As for the social issues, you may be right. I have played on another online card game site where the rating system poisons all social interactions, for the reasons you mention. But I would be curious to know if there are counter-experiences. Does Elo create chronic social problems in online chess, which is considerably more popular than bridge? If not, could it be because the rating is accepted as site-independent and true?

Some of my partners have EBU NGS ratings. NGS ratings are crude and simple but they're easy to understand and friends generally like them. Rating schemes can cause problems and BBO has been set against them from the beginning. In the unlikely event that BBO adopts such a scheme, IMO, BBO should allow dissenting players to opt out and revert to the current daft self-rating system.

[johnu]

If you play ACBL pair games, the scores are recorded at

 

Colorado Springs Power Ratings

 

I can't vouch for their accuracy, but I did find a local cheating pair based on an extraordinary high online rating.

[pescetom]

Yes I have.

They don't care.

They are happy with their archaic little scheme and aren't interested in considering other approaches.

 

In terms of the flaw, at the most basic the NGS scheme calculated results on a session by session basis. It is unable to adjust for the way in which differences in individual boards impact your results. Some board are (naturally) going to be flat. Other have a lot more room for player skill to impact results. If you're unlucky enough to play flat boards versus weak pairs and complicated boards against strong pairs you're going to have a crappy session.

 

NGS, ELO and the like are based on methods that are 40+ years old.

 

In the world of machine learning and AI, 5 years is a lifetime.

 

The EBU's attitude seems to be

 

We came up with the following.

We think it's good enough

There's no reason for us to improve anything

We aren't even interested in doing to bake off to evaluate other schemes or compare accuracy

 

Thanks. Flat boards versus weak pairs and complicated boards against strong pairs is one reason we need to play so many boards at pairs: but if we do and the algorithm evaluates hundreds of such tournaments it's not obvious to me why the rating should be crappy in predicting the outcome of a pair of such tournaments. Many archaic things work. The NGS team say "we expect the standard deviation of the error in your current grade to be around 2%, provided you have a typical mix of partners". Are they way off? Is ELO?

[sfi]

Australia actually adopted an optional ELO-like system about 10 years ago, and our local club decided to start using it. It was in place for two years and was never very popular with the membership. Now that could just be familiarity and it might have been more widely followed in time. However, we also had another club in town who had been using this scheme for many years, and there was some crossover in players between the two clubs.

 

Here the scheme caused real problems. See, the other club was of a lower standard but the initial data had no way to reflect that. So they would come to our club and their rating would drop. Once they realised that, these players would simply stop coming to "protect" their rating. The effect was so pronounced that the club simply dropped the scheme altogether after two years.

[pescetom]

Australia actually adopted an optional ELO-like system about 10 years ago, and our local club decided to start using it. It was in place for two years and was never very popular with the membership. Now that could just be familiarity and it might have been more widely followed in time. However, we also had another club in town who had been using this scheme for many years, and there was some crossover in players between the two clubs.

 

Here the scheme caused real problems. See, the other club was of a lower standard but the initial data had no way to reflect that. So they would come to our club and their rating would drop. Once they realised that, these players would simply stop coming to "protect" their rating. The effect was so pronounced that the club simply dropped the scheme altogether after two years.

 

Sure, the diffusion problem is real, and not just in terms of clubs: just think about two players who only ever partner each other, the system can not distinguish their strength. But there are ways of seeding diffusion and over a few years things should work out anyway - would Meckstroth really put up with me for years, does nobody ever leave that bad club? I figure that if there is so little interplay between clubs that they remain isolated for years then any kind of national or international rating is superfluous to them anyway.

People from a weak club visiting a strong club will finish near bottom more often than not. That will discourage them, with or without a rating system.

[sfi]

Sure, the diffusion problem is real, and not just in terms of clubs: just think about two players who only ever partner each other, the system can not distinguish their strength. But there are ways of seeding diffusion and over a few years things should work out anyway - would Meckstroth really put up with me for years, does nobody ever leave that bad club? I figure that if there is so little interplay between clubs that they remain isolated for years then any kind of national or international rating is superfluous to them anyway.

Yes, it would have worked itself out eventually if people continued playing at both clubs. But the short-term impact was damaging enough both to the players and to the club that the end point was never reached, and the fact that players stopped playing at the other club meant that this point would take longer to reach. Another factor is that people don't like being told they're not as good as they think they are. And this would have been a long-term impact of this rating once it eventually started to reflect reality.

 

The bottom line was that the scheme was measurably hurting table numbers and not bringing any perceived benefit to either the players or the club. So it was dropped.

[hrothgar]

Thanks. Flat boards versus weak pairs and complicated boards against strong pairs is one reason we need to play so many boards at pairs: but if we do and the algorithm evaluates hundreds of such tournaments it's not obvious to me why the rating should be crappy in predicting the outcome of a pair of such tournaments. Many archaic things work. The NGS team say "we expect the standard deviation of the error in your current grade to be around 2%, provided you have a typical mix of partners". Are they way off? Is ELO?

 

Given that the ACBL and the EBU refuse to release data sets or compare the accuracy of the algorithms that are being used, who the ***** can tell...

 

The big issue here is that groups like the EBU refuse to do appropriate due diligence

 

They have a system

They claim it works

[helene_t]

You can judge the progress of your own play by seeing how your score against robots evolve under various conditions (Daylong, with three robots scored against random vugraph deals, etc). Probably your long-term trend will have a negative bias because the robots get better and better, but that will only be a serious problem if we are talking about decades rather than years or months.

 

Judging other players, or comparing your own bridge to that of other players, is something you should avoid, though. It would create lots of social issues:

- players blaming their partners for ruining their rating

- players leaving mid-hand when they are on their way to a result that would ruin their rating

- players selecting partners and opponents based on (mostly misguided) beliefs about how the choice influences their rating

- players avoiding playing when they are tired or distressed because of fear of ruining their own rating

- players accusing each other of cheating

- forum discussions being dominated by conspiracy theories about how the rating system is biased in favour of certain players

- players creating new accounts to start with a fresh rating (which in turns leads other players to be prejudiced against new accounts and thereby make it difficult for genuine newbies to get into the community)

 

Fred and Uday had seen this and/or similar disasters happening on other sites so they rightly chose not to implement a rating system on BBO.

[johnu]

You can judge the progress of your own play by seeing how your score against robots evolve under various conditions (Daylong, with three robots scored against random vugraph deals, etc). Probably your long-term trend will have a negative bias because the robots get better and better, but that will only be a serious problem if we are talking about decades rather than years or months.

 

Judging other players, or comparing your own bridge to that of other players, is something you should avoid, though. It would create lots of social issues:

- players blaming their partners for ruining their rating

- players leaving mid-hand when they are on their way to a result that would ruin their rating

- players selecting partners and opponents based on (mostly misguided) beliefs about how the choice influences their rating

- players avoiding playing when they are tired or distressed because of fear of ruining their own rating

- players accusing each other of cheating

- forum discussions being dominated by conspiracy theories about how the rating system is biased in favour of certain players

- players creating new accounts to start with a fresh rating (which in turns leads other players to be prejudiced against new accounts and thereby make it difficult for genuine newbies to get into the community)

 

Fred and Uday had seen this and/or similar disasters happening on other sites so they rightly chose not to implement a rating system on BBO.

If you play only robot games, it shouldn't matter that the robots get better, or worse, over time. You are playing against other human players so as long as the average field stays the same, your score should vary according to your own skill level trends. If you are an improving player, your scores should go up accordingly.

 

As for the social issues, I don't think a rating system affects the playing environment nearly as much as some people think.

 

E.g. BBO implemented a self-rating system. Many BBO tables (players) state they don't want beginners/novices, or maybe just expert and above. So, many players overrate themselves so they can play in advanced games. Only problem is that it doesn't take more than a hand or two before they are exposed as beginners to intermediate players. Then they'll get bounced from the table. Having a data driven rating system won't make this worse.

 

As for most of the other bad behavior, you see players randomly jumping to 7NT, and redoubling for down many, leaving mid hand, whether or not the hand is going to be a disaster, actual cheating, whether for fun or for actual real life master points, getting banned and creating new accounts, etc. One "solution" would be to "fine" players a rating point of two for egregious bad behavior.

[pilowsky]

All of these problems are the same in real-life (Bridge not being real-life in case anyone forgot) as well whenever there is competition.

Academics are constantly trying to devise new rankings so that they look better than other academics.

citations=likes

number of publications = masterpoints, and don't get me started on the H-index.

[nige1]

BBO implemented a self-rating system. Many BBO tables (players) state they don't want beginners/novices, or maybe just expert and above. So, many players overrate themselves so they can play in advanced games. Only problem is that it doesn't take more than a hand or two before they are exposed as beginners to intermediate players. Then they'll get bounced from the table. Having a data driven rating system won't make this worse.

Understandably, some players prefer self-rating to any objective measure of skill. The nearest BBO gets to the latter is Masterpoints but, unfortunately, that just measures cash resources and experience -- although it would be slightly more effective if totals were automatically "aged".

 

IMO, the frequent necro of this topic shows that there is considerable interest in an objective measure of skill; and a simple optional rating-system might be an acceptable compromise.

[lmhk]

You can judge the progress of your own play by seeing how your score against robots evolve under various conditions (Daylong, with three robots scored against random vugraph deals, etc). Probably your long-term trend will have a negative bias because the robots get better and better, but that will only be a serious problem if we are talking about decades rather than years or months.

 

Judging other players, or comparing your own bridge to that of other players, is something you should avoid, though. It would create lots of social issues:

- players blaming their partners for ruining their rating

- players leaving mid-hand when they are on their way to a result that would ruin their rating

- players selecting partners and opponents based on (mostly misguided) beliefs about how the choice influences their rating

- players avoiding playing when they are tired or distressed because of fear of ruining their own rating

- players accusing each other of cheating

- forum discussions being dominated by conspiracy theories about how the rating system is biased in favour of certain players

- players creating new accounts to start with a fresh rating (which in turns leads other players to be prejudiced against new accounts and thereby make it difficult for genuine newbies to get into the community)

 

Fred and Uday had seen this and/or similar disasters happening on other sites so they rightly chose not to implement a rating system on BBO.

 

In my mind none of the social issues you stated are real problems.

1. blaming partners: If the partners know each other, they need to work it out themselves. The point of competitive gaming is people will care about their rating, and will have to learn how to communicate with each other to improve. Arguments are always going to happen in competitive games. Communication is part of the game. If the partners don't know each other, they should follow site rules on competitive play, such as only being allowed to give constructive criticism rather than saying things like "you suck" or "idiot". Of course, site regulators would be needed to maintain a good environment, and players who are toxic should be penalized by either getting chat banned or banned from competitive play completely.

2. leaving mid-hand: This can be prevented by simply penalizing players who do so.

3. selecting partners: I assume if someone plays in my suggested competitive section as an individual, they should be matched up by the system with a player of a similar rating and cannot choose their partner.

4. fear of ruining rating: Just go play casual. My idea is that the site would be divided into a casual and a competitive section.

5. Cheating/accusing: Cheating might be hard to prevent but at the end of the day, the rating does not give you any actual benefit such as a monetary payment, so there really isn't huge incentive to cheat. (If someone wanted to cheat against me, I couldn't care less. Congratulations on the high rating.)

6.Conspiracy theories: It's just a lot of talk.

7. New accounts: In gaming, I believe the term is "smurfing": if people do this, then surely they will run over their opponents and their rating would increase drastically and quickly, resulting in them matching up with higher rated players soon enough.

 

All in all I think there are too many people being overly worried about the various problems that may arise over such a system, but the fact is the community is soft. A lot of people say "wdp" even though their partner played their hand horrendously and only got lucky. Some people cannot accept any helpful advice/constructive criticism. BBO is almost strictly a bridge club for people to relax and never improve. However, I believe there are still some people who want to play against serious opponents, want to learn, and want to get better by making mistakes, communicating, discussing and accepting advice from others. As stated before, even if such a system were to be implemented, there would still be a casual section for casual bridge lovers, so really no harm done to the former player group.

[hrothgar]

In my mind none of the social issues you stated are real problems.

 

Thats nice

 

In my mind, someone who's been on the forums for a couple days probably shouldn't be explaining the nature of reality to folks with 30+ years experience playing in various online bridge environments.

 

Its nice that you have beliefs

However the rest of us have a whole lot of experience and its says that your assertions are full of *****

[mycroft]

1. "You're dropping my rating". People definitely do care. What happened with Lehmans was that people would only play in very narrow bands - and if you were outside those bands you were either "fishing for lehmans" (which the structure didn't actually do, but who cares about reality?) or endangering my rating (or for those who did understand that a 52 pair needed to make 60% against a 48 pair to not go down, "not fun having to be perfect against the fish"). And heaven help you if you were a 53-47 pair looking for a game.

 

Also, the reason for the rating isn't just so I know how good I am - it's so that I can find someone random that can play. And the "partnership rating" vs "play with randoms rating" can be very different. And now the players protecting their rating won't play with randoms, and especially as a mentor. That's exactly where bridge should be headed.

 

Sure you could play unrated, but you'll never find opponents. When I was mentoring on OKB, we eventually had to find another mentor pair to consistently get a game. It turned out that I ended up inheriting the other student and her regular partner, so my games became tutoring for the whole table. At least we could get a game.

 

2. Can you tell the difference between running and "last hand, get up as dummy so that the next hand doesn't show up?" For Lehmans, your rating was locked in when dummy hit. Conversely, there's the "You think I can't play? I'll torpedo your rating". You think the 7NT ragequitters are going to get fewer when they can actually damage their partner?

 

3. I think I covered that above. Just to say, there's evidence.

 

4. Unrated games - again discussed above. Good luck finding anyone to play.

 

5. It seems that 1% of people cheat (online, can't imagine it isn't the same FtF, just in different ways and harder to catch) now at all levels, where the only thing at stake is the ever-increasing monsterpoint. That won't go *down* when a bad game will cost you. But it won't go up as much as the "whispers" will when random nobody underleads their ace into the only setting defence (never mind the 30% game they had on the other 20 boards, including the other 2 against you).

 

IRL, I'd get one or two "I'm not sure about these guys, ..." a tournament, almost all of which were "they can't play, and this time it was right". The first RAH? I got 6 in 108 boards. And there were 50 directors, many of whom were running 12-16 sessions those 4 days. Again, all that was in games where their rating couldn't go *down*.

 

6. You're right, it is a lot of talk. Look on The Other Site at how obnoxious that talk can already be. As I wrote elsewhere, "well, there are a lot of terms for weaker players, and she’s used them all at least once; so let’s just say - 'can’t play'."

 

7. You can request to reset your rating on OKB. Some people would play for 6 months and request a "re-evaluation" back to 50% each time, because "it was their partner's fault they're a 44" - each time? Really?

 

There are three almost inescapable issues with ELO-ing bridge:

- bridge is a partnership game. You could ELO partnerships; but how do you separate that out? Especially when I, playing with my regular partners, am probably 5% better than me and someone equally rated to my partners, because of the work we've put into the partnership, and the knowledge of our styles and what partner needs from us. This goes double for anyone playing/playing against an uncommon system. My Precision partner and I couldn't get a regular game unless we asked for pairs, because nobody wanted to play with a pickup against Precision.

- you get what you measure. Whatever you put in will be gamed. If I need to keep my rating high, I'll play only with regular partnerships, playing odd systems against pickups, preferably new ones, whose ratings are yet unstable (and likely high). If it's an advantage to have a low rating, sandbagging for handicap is just as easy in bridge as it is in golf (play with randoms against regular partnerships, "don't care" for a week or two,...) And studying the calculation makes this easy.

- what is the rating for? Everyone wants a game with a partner "as good as they are" (so, slightly better), against opponents that are slightly worse. Good luck with that. If it's status, it will be used maliciously (how good someone is already is). If it's for being partnered with randoms, then what good is my ELO (who plays 90% of the time with regular partners against friends)? If it's used to select pro dates, well all I can say is that at that level, it's obvious to everyone at that level where the rest stand - it's what they won and against who. Or their non-playing skills (teaching, not being associated with "less ethical players that win, partly because of those less ethical behaviours", not being an **** to the person paying them) make the difference.

 

It doesn't help that most players don't want to know how good they are. They'd rather keep with their imagination that they're better than that. Prove that they aren't, and they might just go to the next game. Retention is already the #1 problem in bridge. Sure, the younger people that want the serious competition may find this yet another turnoff, but my argument against that is "after a certain point, MPs don't matter, the only thing that matters is what you've won." Let the point counters enjoy it. Everyone knows what I mean when I say one particular Gold Life Master "played at the clubs 6 times a week for 30 years" and another one "has their national win, just needs the other 6000 points". Absolutely noone equates the two.

 

As was said years ago on another one of these threads, the only rating system that will actually work is one that nobody knows is being run, and is used only to partner up reasonable randoms. And it might already be being done, for all we know. It's not like they'd tell you, neh?

[helene_t]

Its nice that you have beliefs

However the rest of us have a whole lot of experience

Exactly.

 

It's not just my experience form StepBridge, other people have similar experience from OKBridge. If someone has experience from other sites that implemented a rating system and observed more positive than negative social implications of the rating system, I would find it interesting. But the belief that BBO would be better with a rating system is not so interesting. I'm sure the World would also be a better place if we had unicorns instead of horses.