Rating Players

[Tola18]

(OK, there may be good players who would be willing to play robot award using someone else's account to boost the client's rating and getting payed for it, but I don't think that would be so widespread).

 

People could still manipulate the system by creating multiple accounts, playing some robot awards with each, and then continue with the one that got most lucky. But the money and time that would cost would deter most from doing so.

Sure, but the same can prob be said about almost any rating system. Unless perhaps if you must play with your own name and own face.

 

So, for the time being, your suggestion for using robot tournaments as rating help, IF we really need a rating system, is very interesting, I think.

 

I second this proposal.

[Wackojack]

Helene's idea of using robots for rating looks attractive. Suggestion:

 

1. Use robot reward format with imp scoring and random hands (not best)

2. Free admission.

3. Participants names are hidden min number say 30.

4. To get onto the ladder play a certain number of tournaments such that the level of confidence is high. 200 hands say looks reasonable Mathmaticians to work this out.

5. No limit to number tourneys that can be played.

6. Rolling expiry of say 2 years.

 

One problem that I can see is that GIB plays 2/1 exclusively, so those that don't play this will be disadvantaged and so will be discouraged from participating. Another is that the more you play with GIB the more you learn to adjust to its foibles. So no doubt the more you play in these tournies the better your rating will get.

[helene_t]

1. IMP scoring against the field doesn't work for robot reward as they are not duplicate. IMPing against PAR would be fine. IMPing against passout might be better than total points or money won. Note that it is essential that they are not duplicated to avoid cheating.

 

2. Hmmmm not sure if this would be in BBO's interests! Currently the fee is 25 $cents for 25 minutes for robot race but I think robot reward and robot rebate should be included as well.

 

3. This doesn't matter since they are not duplicated.

 

4. Yeah or maybe you start with a prior belief that your percentile relative to the whole ladder is [0;100] and then a 95% credibility range is reported. In that way you can see on someones rating not only the estimate but also how uncertain it is. Someone new to BBO will have a range [2.5;97.5]. An average player with 1000 of recent boards may have [49;51].

 

5. Agree

 

6. Yeah agree, although an exponential forget is cooler, roling expiry of two years is easier to understand.

[Wackojack]

Comparison against par is capricious. Is it really essential for boards not to be duplicated? Suppose over the tournament, duplicated boards were played in a random order for each player and had no number and had no player identification. Then the scope for cheating would be virtually zero. I believe the best assessment of ability is to compare with other players.

[helene_t]

Yes I think it is essential that boards are not duplicated. Not only to avoid cheating but also to avoid the issue of whether or not results should be adjusted for the strength of the field, and if so: how.

 

Obviously IMPing against a table with four robots would be better than IMPing against par.

[hrothgar]

Comparison against par is capricious. Is it really essential for boards not to be duplicated? Suppose over the tournament, duplicated boards were played in a random order for each player and had no number and had no player identification. Then the scope for cheating would be virtually zero. I believe the best assessment of ability is to compare with other players.

This seems nonsensical

 

1. You don't base a rating system off of a single tournament. You need a large corpus of hands. Its impractical to have players complete a meaningful hand sample in the single event. Equally important, you really don't want to be using the same hands for multiple events stretched across multiple days.

 

2. GIB uses an intrinsically stochastic process for decision making. Even if players use the exact same boards, there is no guarantee that they will face the same bidding / play decisions.

 

When it comes to rating systems, duplication is both unnecessary and undesirable.

[helene_t]

I think you exagerate, Richard. In non-duplicate scoring, probably most of the variance between players on a single round is to be attributed to differences between the deals played, and it would be nice to get rid of it.

 

But IMHO it just doesn't trump the arguments for having non-duplicated boards. And I suspect that IMPing against PAR would remove most of the unwanted variance.

[hrothgar]

I think you exagerate, Richard. In non-duplicate scoring, probably most of the variance between players on a single round is to be attributed to differences between the deals played, and it would be nice to get rid of it.

I don't disagree with this; however, I don't think that the single round criteria is in any way meaningful to a rating system.

 

I'd be leery about using a single day or event a single event for a meaningful long term rating.

[helene_t]

hey richard, of course a single round is not meaningful. Jack suggested 200 rounds. Then the ratio between the between-player variance and the residual variance improves by a factor 200, fine. But that is true regardless of which method you use, so the duplicated results over 200 boards will still be much better than non-duplicated results over 200 boards. There is a reason that all serious competition is duplicate.

[Wackojack]

Comparison against par is capricious.  Is it really essential for boards not to be duplicated?  Suppose over the tournament, duplicated boards were played in a random order for each player and had no number and had no player identification.  Then the scope for cheating would be virtually zero.  I believe the best assessment of ability is to compare with other players.

This seems nonsensical

 

1. You don't base a rating system off of a single tournament. You need a large corpus of hands. Its impractical to have players complete a meaningful hand sample in the single event. Equally important, you really don't want to be using the same hands for multiple events stretched across multiple days.

 

2. GIB uses an intrinsically stochastic process for decision making. Even if players use the exact same boards, there is no guarantee that they will face the same bidding / play decisions.

 

When it comes to rating systems, duplication is both unnecessary and undesirable.

1. I was not suggesting a rating system off a single tournament. I was suggesting how in a single tournament, cheating might be avoided.

 

2. Thats why I was suggesting that the rating is based on comparison with humans and not with par.

 

Comparing with par even when imped would need many times more hands to base a rating than comparison with humans or indeed with other GIBs.

[hrothgar]

hey richard, of course a single round is not meaningful. Jack suggested 200 rounds. Then the ratio between the between-player variance and the residual variance improves by a factor 200, fine. But that is true regardless of which method you use, so the duplicated results over 200 boards will still be much better than non-duplicated results over 200 boards. There is a reason that all serious competition is duplicate.

Hi Helene

 

I don't think that we're actually disagreeing...

 

If I thought that it were possible to use duplicated boards in a meaningful way for online bridge ratings, I would prefer to do so.

 

I don't think that duplicated boards can be used for a broadly deployed system. Furthermore, i don't consider this particularly troublesome.

 

A system based on nonduplicated boards will require a larger sample to generate the same degree of accuracy. Such is life.

[Wackojack]

As evidence, this is my experience with MP robot tourneys where you play 8 boards for 25 cents and your results are matchpointed against other humans partnering robots playing the same boards.

 

Started about 2 years ago and now played 1080 hands. My playing in these tourney's now has tailed off to about one 8 board tourney a week. Over the last 500 hands played, the 128 hand moving average and the overall average from the start has stabilised to within a 0.2% band on a % MP scale and at the present time they are exactly equal.

 

What I think this tells me is that my % record accurately reflects my playing ability at MP relative to the rest of the field that plays in these tourneys. What of course it does not tell me is how good the field is relative to the average BBOers or good tournament players.

[spotlight7]

Hi:

 

Rating new players using 'hands played' may not be wise.

 

Mike Cappelletti had one of the top teams in the Washington D.C. area several decades ago. They played a 28 board IMP match against a team with zero master points and lost to the newbies.

 

The 'new team' could play bridge, they just did so in their own circle which was quite good.

 

What is the problem? I normally know how good a person is after a few hands.

Do you really expect to find that many good players as a 'pick up player' on bbo?

 

Label people as friends if they play at/near your level. Avoid playing with players that your notes list as 'butcher' or 'wild overbidder.'

 

Regards,

Robert

[eyhung]

As I stated in another thread, I think it's far more important to rate people on the basis of their behavior rather than skill. In particular, the unpleasant behavior of quitting in the middle of the hand is easily quantifiable and can be discouraged with a reputation system that tracks how many times a player has left prematurely in the last 50 hands played, and allowing main room servers to specify how tolerant they want to be (bar people with more than a 10% drop rate, for example).

 

Right now the main room of BBO is trending towards the unpleasant nature of Yahoo! bridge with people popping in, seeing a bad hand, and leaving, or leaving immediately when their partner makes what they consider a bad mistake. I think this is fine for the Relaxed Club, but the Main Room should be a little more serious. It's my belief that this behavior has induced most of the good players on BBO to only play in set games, tournaments, or team matches.

[cloa513]

What if you have to quit or the location you have has irrattic internet?

[eyhung]

It's a rolling average so your quit will be gone in 50 hands, and 1 quit in 50 hands will give you a 2% "drop" rate, which should be enough to satisfy most hosts. I suggest making the ban on drop rate completely customizable by the server, with a default of 5-10%. If you are disconnected a lot, well, I don't think most hosts would like that, so you would have to play in Relaxed Club where there are no restrictions, or find a host who can tolerate your frequent drops.

[Doraem0n]

In another website where they offered ratings, players seldom abandon a hand midway because to default immediately impacts their rating negatively. The most annoying thing I experienced in BBO is people leaving midway through a hand.

[helene_t]

Here is a proposal for people who have Bridge Browser:

 

Find out which rating system is most accurate, by means of cross-validation. If you remove a single board* from the database, a good rating scheme is one that can predict the IMP or MP score on that board, from a statistical analysis of the remaining boards.

 

There is an issue with partnership rating vs individual rating. So an alternative project could be: remove all hands played by a particular partnership and try to predict their performance from an analysis of the remaining boards (in particular those that they played with other partners).

 

Such a study would give clues to how accurate a rating based on total points would be relative to one based on IMP relative to PAR or to IMP relative to the hand played by robots, or to IMP relative to the BBO field.

 

*by "board" I mean board x played by players a,b,c,d, i.e. not all the times board x was played, just one of them. You do this for many (ideally all) boards in the database, then compute the average prediction error.

[djeast]

BB will never be a truly viable site as long as it is unable to have a regulated and sophisticated system of awarding colored points, at least to the silver level, and combine a rating system with ACBL ofline play. There needs to be a value to the money spent online, and no penalties for playing there. Colored points earned can be calculated very easily with the promotion in value of Gold, Red, and Silver points, and the devaluation of Black and Uncolored online points. The rating system if developed should be in addition to the current online rating. Opting in or out should not exist. BBO doing a terrible job at evaluating the quality of players. For those of us who need colored points, without an ability to enter games with correctly rated players for self improvement, BBO has nothing for me. I'll save my money and take up knitting.

[djeast]

Cheating in BBO speedballs is BBO's fault and no one else's. All Speedball tourneys need to be open seating, no partnerships, and after BBO develops an accurate Player Rating system in conjunction with ACBL, stratified play with players of similar talents playing agaist each other.

[hrothgar]

BB will never be a truly viable site as long as it is unable to have a regulated and sophisticated system of awarding colored points, at least to the silver level, and combine a rating system with ACBL ofline play. There needs to be a value to the money spent online, and no penalties for playing there. Colored points earned can be calculated very easily with the promotion in value of Gold, Red, and Silver points, and the devaluation of Black and Uncolored online points. The rating system if developed should be in addition to the current online rating. Opting in or out should not exist. BBO doing a terrible job at evaluating the quality of players. For those of us who need colored points, without an ability to enter games with correctly rated players for self improvement, BBO has nothing for me. I'll save my money and take up knitting.

Comment 1: There are 15K people logged in right now. Feels pretty viable to me...

 

Comment 2: BBO doesn't cost anything, so its hard to understand how knitting is going to save to any money

 

Comment 3: You seem to be laboring under the belief that there is some relationship between "attendance points" and skill. If this is the case, then you are probably beyond hope.

 

Comment 4: BBO ain't the ACBL

[nigel_k]

I don't know the answer to the rating issue but one way might be to just outsource it, i.e. make the hand records easier to query externally so people can build their own rating system and make the results available on their own site. Over time, some would become popular and some would disappear. This wouldn't allow a tournament organiser to auto-exclude people based on rating but would be fine if you just want to restrict the people at your table.

[matmat]

Cheating in BBO speedballs is BBO's fault and no one else's. All Speedball tourneys need to be open seating, no partnerships, and after BBO develops an accurate Player Rating system in conjunction with ACBL, stratified play with players of similar talents playing agaist each other.

LOL

 

Now, i am not sure about this, but i was under the impression that bridge was a partnership game.

 

Excluding pairs that play and practice together (THE VAST VAST VAST VAST VAST VAST VAST VAST (that's a lot of VASTs) majority of who DO NOT CHEAT) is ridiculous. it randomizes the field, causes bad and high variance bridge and rewards good luck rather than good actions.

 

This and your previous post make it pretty clear that you are exactly the sort of person I prefer not to have to meet at a bridge club; someone who loves to result, blame others, and argue. how's that masterpoint attendance award coming along?

[bid_em_up]

I'll save my money and take up knitting.

Promises, promises.

[jillybean]

I'll save my money and take up knitting.

Promises, promises.

LOL