ACBL Performance Rating

[jmc]

Is anyone else excited about ACBL'S CEO Jay Baum's anouncement of a new performance rating of some kind being used by the ACBL? In this month's Bulletin he said:

 

In the coming year, we will move closer to a system that will allow us to rank players according to playing ability measured in the short term. We are still working on details- and the system would apply only to tournamnets initially- but a players rating might be number of masterpoints earned in a certain period divided by number of sessions played.

 

What do you think? Will it work? Will it increase or decrease the number of players. How will it change strats and flights? Should it resemble a chess elo, a batting average, a golf handicap? Anyone have inside info? All comments welcome.

 

jmc

jonathan.cohn@gmail.com

[mike777]

This may be a way to fight back against online play.

 

The problem is for many players online bridge is becoming the norm except for the very top players or very old local players.

 

I bet even the top players play "more" online hands.....

 

Keep in mind online points count for nothing in the races.

Are online points going to count for anything in this rating system?

In another post the feeling was online ACBL tourneys are worse than the local club by a large margin.

[Vilgan]

This may be a way to fight back against online play.

 

Why would they want to fight back against online play? Okay, if someone wants to play all day and get lets say 4 colorless points that show up on no ranking list whatsoever... who cares? It effects absolutely nobody else whatsoever. Others who go to tournaments can rack up 10-20 a day and be involved in a more social/fun atmosphere.

 

To be honest.. I think some sort of rating system would be good, but it should not be based on masterpoints whatsoever. Perhaps people are rated.. then play in a flighted event, and it might tweak their rating up or down a tiny bit depending on how they did compared to how they were expected to do. Masterpoints are a marketing tool... and a great one. But they should not in ANY way be utilized in an actual rating system that is meant to be accurate.

 

Eric

[paulg]

The EBU masterpoint scheme is similar to the ACBL and a player accumulates club (black) points and national (green) points endlessly. A player's rank is decided by a mix of these points and, unless young, the rank is more of a lifetime achievement award.

 

A few years ago the EBU introduced its Gold Point list. Gold Points, which depreciate in value over a five-year period, provide a measure of form for more senior players. Gold Points are awarded only when you win 5 or more National Points in a single event: for example, if you won 5 National Points in an event, then you'd get 1 Gold Point that would depreciate over the next five years; if you won 8 National Points, then you'd get 1.6 Gold Points.

 

5 National Points is approximately equivalent to 30 golds in the ACBL.

 

The Gold Point list is used for ranking in a couple of events (we don't have any KO Brackets here) and for entry to the trials system.

 

Paul

[jmc]

I understand why they want to make the new rating system connected to masterpoints. MPs are familiar to players and a ratio of MPs won to events played will make some sense of the number. It fails in some respects however. If I were to pick a system out of thin error I might instead base it one something like the system chess uses.

 

One's performance would be compared opposite the quality of the opposition. The better one did, then the better or more rating points would be won. I am next an expert in these things but a similiarity to chess elo would be handy for those familiar with both systems.

 

jmc

[kenrexford]

The subtlety I noticed was that the new system is apparently designed to enhance the ability of the older player, who has "lifetime achievement" but lessening competitive edge, to "play down" because of recent success rarity. It will apparently allow those with "too many" points to play down into the B or even C flights.

 

This may help those folks. However, I suspect the opposite. I imagine that many of the "new money" folks in the B flight will start getting real upset real fast at the "tired pros" dropping into their events and taking all of the masterpoints, or at least the perception that this is happening will be there.

 

I am also concerned about the "ratings" as affected by people who play with pay dates or with rookie friends/family. Unless the ACBL is prepared to adopt some form of adjustment for playing down value, and somehow guesses right as to how much to adjust, the end result seems to be that people will be encouraged to not "play down" from the top levels, but more inducement to "play up" will be there.

 

All that said, the existing system has been flawed from the pride perspective. Your masterpoint total is a derivative of skill, but also of where you live, how often you play, and how long you have played. This new guage might appeal to some.

[hrothgar]

Rating systems are a horribly complex topic that we've discussed several times in the past. Here's a quick summary of my views. I can expand upon this if anyone cares.

 

In order for a rating system to have any value it needs to accomplish two very disparate goals:

 

1. It needs to be accurate

2. End users need to believe that its accurate. In turn, this typically requires that end-users can understand how the rating system is calculated. Worse yet, it requires that people accept the fact that the rating system will rank roughly half of all bridge players as below average..

 

I don't think that its possible to simultaneously achieve both sets of goals. If people are comfortable with how a rating system is derived its typically too complicated to have any value. (This is the problem that the ACBL is currently experiencing with Master Points. Masterpoints are very problematic for seeding purposes). Alternatively, I think that it is possible to create fairly accurate rating systems. However, people aren't going to be happy with it.

 

I suspect that the best way to proceed is to try to minimize the problems. The number of occasions that you actually need an accurate rating system is pretty small. Many of these situations are major events like the Spingold. You might have better luck trying to introduce an accurate system for use in the top flights of competition and then let it trickle down to the masses if they actually seem to want this type of system. In the mean time, its probably best to design conditions of contest where accurate rating systems don't really matter.

[pclayton]

Rating systems are a horribly complex topic that we've discussed several times in the past.  Here's a quick summary of my views.  I can expand upon this if anyone cares.

 

In order for a rating system to have any value it needs to accomplish two very disparate goals:

 

1.  It needs to be accurate

2.  End users need to believe that its accurate. In turn, this typically requires that end-users can understand how the rating system is calculated.  Worse yet, it requires that people accept the fact that the rating system will rank roughly half of all bridge players as below average..

 

I don't think that its possible to simultaneously achieve both sets of goals.  If people are comfortable with how a rating system is derived its typically too complicated to have any value.  (This is the problem that the ACBL is currently experiencing with Master Points.  Masterpoints are very problematic for seeding purposes).  Alternatively, I think that it is possible to create fairly accurate rating systems.  However, people aren't going to be happy with it.

 

I suspect that the best way to proceed is to try to minimize the problems.  The number of occasions that you actually need an accurate rating system is pretty small.  Many of these situations are major events like the Spingold.  You might have better luck trying to introduce an accurate system for use in the top flights of competition and then let it trickle down to the masses if they actually seem to want this type of system.  In the mean time, its probably best to design conditions of contest where accurate rating systems don't really matter.

I'm less concerned about accuracy (how would you define 'accurate' in this context anyway), however:

 

A. If a couple with 4,000 MPs has lost a step (and their record over the last several years reflects this) and wants to play 'down', I'm fine with that.

 

B . If my teammates have had some success over the last several years (but have a lot less MPs than said couple) and want to play up then the league should encourage this.

 

If the legaue institutes some sort of 'current' rating system that accomplishes A and B, then I think its a worthwhile endeavor.

[Al_U_Card]

Maybe we should call in Bill James (The SABR metrician) who came up with all kinds of neat baseball measurement stats to determine playing ability etc. Some cool calculation (uhhhh how about the 3F method) that takes into account, say:

 

Frequency, how often you play

 

Field, how many people you play against

 

Force, the strength of yourself and your partner relative to the average opponent

 

This could generate a running average of some kind that would relate to your success rate.

 

Anything is possible... :)

[hrothgar]

Maybe we should call in Bill James (The SABR metrician) who came up with all kinds of neat baseball measurement stats to determine playing ability etc. Some cool calculation (uhhhh how about the 3F method) that takes into account, say:

 

Frequency, how often you play

 

Field, how many people you play against

 

Force, the strength of yourself and your partner relative to the average opponent

 

This could generate a running average of some kind that would relate to your success rate.

 

Anything is possible... :)

I actually spent a fair amount of time looking into this subject a few years back. (OKB was thinking about reworking their ratings systems and moving from the Lehman's to something else)

 

From my perspective, the most fruitful way to approach this problem was to using signal processing techniques (a discrete time Kalman filter looked like a good starting point). However, I quickly became convinced that the real problem was political rather than technical. The debates on the OKB mailing list trying to describe a relatively simple algorithm like the Lehman system was hideous. The Lehman system was simple enough that folks could perform the calculations by hand and had been automated in a number of spreadsheets. it was still too complicated for people to trust. Signal processing systems can get quite complicated. The thought of trying to explain anything like this to Carl Hudechek made my hair turn grey. I quickly decided that rating systems were more trouble than they were worth.

 

I'll throw out one quick observation about the SABR system. The SABR metrics system seems to be a very useful way to evalate players. ("Moneyball" certainly makes a persuasive case). However, SABR is a system that outsiders (managers, owners, fans) use to evaluate the performance of professional ball players. There is a lot of money to be made recognizing underpaid and overpaid baseball talent. For the most part this market doesn't exist for bridge players. (There is a market for top pros, but its pretty thin).

 

In contrast, organized bridge is a social activity. People play to have fun. If you create an accurate rating system its going to tell a lot of people that they're below average. I'm not sure that people want to know that. In theory you can finesse the issue. (Only rate the top 25% of players and let the rest believe that they're somewhere between 50% and 75%). Even so, I just don't believe that there is that much demand for a good rating system

[awm]

There are several pretty serious problems that could be helped by an accurate rating system. In particular:

 

(1) There are a number of people who have been playing bridge a really long time but aren't really that good. They enjoy playing regularly at their local club, but often don't want to attend tournaments, because they will be thrown into flight A or bracket one or whatever with the elite players due to having accumulated five to ten thousand points in their lifetime. I know a number of people who choose not to participate in tournament bridge for exactly this reason, and would be happy to play in some medium-flight game.

 

(2) There is a set of up-and-coming younger players who would like to compete against the best. Excluding those who play full-time as pros, most of these younger players simply haven't had the time to accumulate very many masterpoints. They would like to get as much experience playing against the best players as they can, but right now this opportunity rarely exists outside national tournaments (which require a lot of time and money to attend, also difficult for young working folks). Even though a lot of top players may show up to the local regional, they're usually playing in bracket one knockouts (and not the open pairs), which have a steep masterpoint requirement to get in (and directors almost never let people play up).

 

(3) There are a number of older experts who were international-caliber competitors in their prime but no longer play to that level of skill. A lot of these experts spend a great deal of time teaching, and thereby giving back to the game. It would be nice if they had the opportunity to play with their pupils in tournaments, to give the newer player a taste of tournament play (probably this would be good for bridge in general, both encouraging new tournament players and bringing out the older big-name players to tournaments). However, as things currently stand the combination of stratification/flighting being based on the maximum points in the pair and the fact that masterpoints only accumulate (meaning a once-great player has a massive number of them even if well past their prime) means this can only happen in bracket one (or open flight) which reduces the frequency. I know a number of players who were once world class (but are getting older now) who still teach the game and show up regularly at clubs, but never make it to the local regional.

[pigpenz]

i feel that you know where you belong!

if you want to play against the best you should be able too unless it is a restricted event...which there are almost none left...being MASTER/Non MASTER Pairs. Pick up a bullettin from the 70's or early 80's and look at the events listed from a Regional.

 

Mens/Womens Pairs

Master/non Master Pairs

Open Pairs

Knockouts

Swiss Teams

Open Pairs Qualifying/Final

 

the scheduale for todays regionals

is almost entirelly Flighted KO's several times

a day and variuous pairs/senior pairs games.

[Al_U_Card]

Yah, like I offered before, on the 0 to 100, Dead to Meckwell scale, I put myself at a 55......

[jmc]

It has almost been a year and this "new rating system" is still only pie in the sky. I am disapointed that the announcement made it seem imminent and nothing has happened. Vote of no confidence for the ACBL's lame duke announcement.

 

jmc

[pbleighton]

I think it's a complete waste of time.

 

Peter

[hotShot]

I actually spent a fair amount of time looking into this subject a few years back. (OKB was thinking about reworking their ratings systems and moving from the Lehman's to something else)

 

From my perspective, the most fruitful way to approach this problem was to using signal processing techniques (a discrete time Kalman filter looked like a good starting point). However, I quickly became convinced that the real problem was political rather than technical. The debates on the OKB mailing list trying to describe a relatively simple algorithm like the Lehman system was hideous. The Lehman system was simple enough that folks could perform the calculations by hand and had been automated in a number of spreadsheets. it was still too complicated for people to trust. Signal processing systems can get quite complicated. The thought of trying to explain anything like this to Carl Hudechek made my hair turn grey. I quickly decided that rating systems were more trouble than they were worth.

There is a much simpler way to get a good rating, than signal processing.

 

Set the start rating of a player to 2000.

The rating of a pair/team in the arithmetical average of the players involved.

 

At the end of a tourney you do this simple math:

 

Any player/pair/team that scored worse than you, but has a ranking value better or equal to yours, increments your ranking by 1/2/4 points.

 

Any player/pair/team that scored better than you, but has a ranking value lower than yours, decrements your ranking by 1/2/4 points.

 

Simple to explain and it has some nice side effects.

1) Playing the same group of people the ranging will stay stable, winning against weak player does not help your ranking.

2) If player get better or weaker, there score will rise or drop with their results.

3) Loosing against better player does not hurt your ranking.

4) Large tourneys are more selective than small ones.

5) Regional and national tourneys will adjust the rankings between clubs.

 

 

Problems:

If everybody starts with a ranking of 2000, a local ranking will be quite stable after 5-10 tourneys, but to establish a regional or even national adjustment,a lot tourneys will be needed. This problem will be smaller if results of previous regional and national tourneys will be used to determine start values for the ranking points.

Club player that have no scores from regional or national events will get a starting value a little below the average of those club players who have a score.

[joshs]

I actually spent a fair amount of time looking into this subject a few years back.  (OKB was thinking about reworking their ratings systems and moving from the Lehman's to something else)

 

From my perspective, the most fruitful way to approach this problem was to using signal processing techniques (a discrete time Kalman filter looked like a good starting point).    However, I quickly became convinced that the real problem was political rather than technical.  The debates on the OKB mailing list trying to describe a relatively simple algorithm like the Lehman system was hideous.  The Lehman system was simple enough that folks could perform the calculations by hand and had been automated in a number of spreadsheets.  it was still too complicated for people to trust.  Signal processing systems can get quite complicated.  The thought of trying to explain anything like this to Carl Hudechek made my hair turn grey.  I quickly decided that rating systems were more trouble than they were worth.

There is a much simpler way to get a good rating, than signal processing.

 

Set the start rating of a player to 2000.

The rating of a pair/team in the arithmetical average of the players involved.

 

At the end of a tourney you do this simple math:

 

Any player/pair/team that scored worse than you, but has a ranking value better or equal to yours, increments your ranking by 1/2/4 points.

 

Any player/pair/team that scored better than you, but has a ranking value lower than yours, decrements your ranking by 1/2/4 points.

 

Simple to explain and it has some nice side effects.

1) Playing the same group of people the ranging will stay stable, winning against weak player does not help your ranking.

2) If player get better or weaker, there score will rise or drop with their results.

3) Loosing against better player does not hurt your ranking.

4) Large tourneys are more selective than small ones.

5) Regional and national tourneys will adjust the rankings between clubs.

 

 

Problems:

If everybody starts with a ranking of 2000, a local ranking will be quite stable after 5-10 tourneys, but to establish a regional or even national adjustment,a lot tourneys will be needed. This problem will be smaller if results of previous regional and national tourneys will be used to determine start values for the ranking points.

Club player that have no scores from regional or national events will get a starting value a little below the average of those club players who have a score.

Well I am not sure this gives a very good ranking:

Imagine 4 pairs playing, A B C D, all starting even:

Event 1:

A wins

B 2nd

C 3'rd

D 4'th

 

So the Ratings are now: +3, +1, -1, -3

 

Event 2:

Same result as event 1

 

Ratings stay the same: +3, +1, -1, -3

 

Event 3:

Again the Same

 

Ratings still at : +3, +1, -1, -3

 

Event 4:

B wins

A 2'nd

D 3'rd

C 4'th

 

Ratings are now: +2, +2, -2, -2

 

So even though A beat B 3/4 times and C beat D 3/4 times there ratings are the same.

[mycroft]

This scheme, like all rating schemes that try to level individuals, suffers from the Pair Problem and is exploitable that way.

 

Find a player who is pretty good, but plays club bridge with anybody. That person's rating will drop relative to their ability. Establish a partnership with that player, and get another pair of "play ups" for your team. You will play, simply because of established partnerships, significantly better than your rating. And because my pair rating with this player is significantly lower than my rating, but my rating goes up the same amount as his does when we beat a "stronger" pair,...

 

It's harder to exploit than the Bracket 7 game (find a pair of decent up-and-comers and play with them. It'll drop you a bracket or two (or 5 or 10, if you're in Gatlinburg), and you should win. We have someone who exploited this for years - now he doesn't, but now he has 1000-odd masterpoints and it doesn't play as well any more), but anything that doesn't reflect the fact that my (relatively weak, in general) Precision partner and I play miles above any rating we would individually have, in an environment where you are trying to rate the individual as a prospective (pickup) partner, will be fundamentally flawed and exploitable.

 

In other words, Meckstroth-anybody is a formidable pair, but Meckstroth-Hamman playing pickup against Sontag-Weichsel? I know where my money lies.

 

Michael.

[TimG]

In the coming year, we will move closer to a system that will allow us to rank players according to playing ability measured in the short term. We are still working on details- and the system would apply only to tournamnets initially- but a players rating might be number of masterpoints earned in a certain period divided by number of sessions played.

If it involves masterpoints, the problems will surely persist.

 

I'm surprised no one who has posted here is old enough to remember that there once was a way for those who had accumulated a bunch of masterpoints to play down. I think it was based upon a simple average of masterpoints won in the last five years. Flight A players could play down as a result of winning relatively few masterpoints in recent years.

 

To Richard's mention of the "SABR system": SABR is the Society for American Baseball Research. There is no official SABR approved system of evaluating performance. SABR members do tend to like to use statistics to evaluate players and tend even more towards liking to talk about the results of statistical studies. But, they might be using OPS or Adjusted OPS or OPS+ or Grey Ink or Black Ink and won't be kicked out of the society for using one and not the other.

[mike777]

We can always bring back the method one of my older partners told me about.

If you win the Spingold you win something like 10 MP but at the start of next year you always lose ten MP no matter what you did last year. He did make LifeMaster under this system. :lol:

[awm]

Designing a good rating system is a pretty complex problem even if we assume we are rating pairs rather than individuals.

 

If we rate "masterpoints per session" then this deters people from playing in club games. After all, even if you win a small club game you might get something on the order of 0.5 or 0.6 masterpoints. Top players average a lot more than this per session playing in regionals. So this would mean that if (say) Fred were to play in a club game his rating would go down even if he wins. Of course, the alternative of rating "masterpoints over masterpoints possible" favors people who routinely win in weak club fields (getting 0.5/0.5 possible masterpoints) over people who do well but don't necessarily win in top flight events (say obtaining 22/150 possible points in LM pairs).

 

Things are made even more difficult by the observation that beating down bad players is not exactly the same skill set as doing well in a strong field. There's a pair at my local club that wins almost every time they play (occasionally someone has a big game and drops them to second or third). This is because they know all the club regulars and how to exploit their weaknesses (who to overbid against, who to double, etc). Looking at their scores, they must average about 95% of the matchpoints against the weak pairs and less than 50% against the good pairs! In any case, there are other pairs at the club who routinely do respectably in regional or national events (and do well against this pair when at their table) who are probably "better" than these two, yet the club standings would indicate otherwise... The point being that there are pairs who can score in the mid-50s routinely in LM pairs who do only slightly better in the local club game, whereas there are other pairs who would be totally out of their depth in LM pairs but can score over 60 in the local club game routinely.

 

This is not true of chess, where any player who can beat grand masters will win routinely at his local chess club.

[jmc]

I agree that rating systems are very complex. They can also have some unintended consequences. i.e. there are chess players that play just enough to keep their ratings fresh, but not too much because they don't want to risk dropping below 2000 or 1900 or whatever. I'm sure the ACBL does not want to discourage people from playing in any way. BUT...

 

My current concern is that the ACBL announced a system was coming. The announcement made it sound forthwith. Then it disapeared. This is neither professional nor proper. ACBL members spend dues and entry fees to have a quality run organization. If it wasn't ready, they should have announced they were considering it, or entertaining proposals, or <gasp> not announced it at all.

 

jmc

[geller]

There are two functions of a rating system. One is to provide rankings of players (and from the point of view of the national federation) to incentivize players to play more. The flaws of the masterpoint system have been amply discussed in this thread and elsewhere, as have the difficulties in coming up with something demonstrably better. I don't have anything new to say about this.

 

But there is also a need for a rating system for flighting. This doesn't have to give am exact rating; just a rough and ready system is OK. The system we use in Japan is as follows. Each player has a "seeding point" (SP) total updated monthly. This number is computed from the player's masterpoints (MP) as follows

 

SP = 0.05 * min(career MP total, 3000) + 0.5 * (MP won in last 12 months)

 

The SP of a pair is the total of the two players, and the SP for a team is the sum of the four top players. Pairs or teams are flighted by SP totals. Different countries would want to tinker with the constants in the above formula based on the details of their MP system, but the above seems to provide a workable flighting system that almost all players are happy with.

[hotShot]

Well I am not sure this gives a very good ranking:

Imagine 4 pairs playing, A B C D, all starting even:

Event 1:

A wins

B 2nd

C 3'rd

D 4'th

 

So the Ratings are now: +3, +1, -1, -3

 

Event 2:

Same result as event 1

 

Ratings stay the same: +3, +1, -1, -3

 

Event 3:

Again the Same

 

Ratings still at : +3, +1, -1, -3

 

Event 4:

B wins

A 2'nd

D 3'rd

C 4'th

 

Ratings are now: +2, +2, -2, -2

 

So even though A beat B 3/4 times and C beat D 3/4 times there ratings are the same.

The question is, how significant is 1 point?

If you think of a bigger tourney with 100 tables, the rankings will be between

+400 and - 400. I expect that values of 3500+ for world class player and 500 for beginners will be reached over time. I would consider less than 10 points as not very significant.

 

In your example A and B won 4/4 times over C and D.

A was 4-6 points better than C/D

B was 2-4 points better than C/D

 

A's ranking was between +3 and +2.

B's ranking was between +2 and +1.

C's ranking was between -1 and -2.

D's ranking was between -2 and -3.

 

This means that the ranking is quite stable, within this group.

 

Now assume that B came from some other club and has ranking up to +20. AC and D have a ranking of 0.

After your first event: the rankings will be:

+3, +19, 0, -2

After the 2nd event:

+4, +18, 0, -2

After the 3rd event:

+5, +17, 0, -2

After your 4th event:

+5, +17, -1, -1

 

So ever a longer period of time A and B will get to the same level, although A is not playing outside the club. And because B is loosing points to A in this club, B can regain points playing at the other club.

[barmar]

My current concern is that the ACBL announced a system was coming. The announcement made it sound forthwith. Then it disapeared. This is neither professional nor proper. ACBL members spend dues and entry fees to have a quality run organization. If it wasn't ready, they should have announced they were considering it, or entertaining proposals, or <gasp> not announced it at all.

IIRC, all they announced was that they were starting to work on it, not that it was around the corner. Furthermore, they pretty clearly stated that the first pass would be targeted only at high-level events (which to me suggests that it might be an adaptation of the way they do seeding points for NABC+ events). In time they may expand it to all players.