EBU National Grading Scheme

[phil_20686]

So, today the NGS went live, and you can look up EBU members by name to find out how good they are...apparently.

 

The NGS ranks you according to the MP score you would expect if you were to play in a field of the whole EBU. Supposedly. So I was interested about how accurate we would expect this to be? This is the primary question I would like ppl to answer.

 

You can view all EBU members here if you want to look up anyone.

 

For a sample of current accuracy, these are the scores of those BBF posters who are EBU members (known to me) and post under their real names*.

 

Frances Hinden:68%

Me: 64%

Andy Bowles (gnasher): 57%

gordontd: 58.4%

JAllerton: 64%

 

These alone seem to indicate that it is not very good yet, as andy seems much underrated, and I seem much overrated, but i suspect many of us have not yet reached close to the thousand ranked boards that they claim to need. Think I am about 600 according to my ebu members records.

 

*Since you post under your real names, and the NGS rankings are public, it didnt seem like anyone would mind, if you do mind let me know and I will edit you out, hopefully before too many ppl see it.

[wank]

well i don't play club bridge very often so it'll probably take a while for me to reach 1000 boards.

[Finch]

well i don't play club bridge very often so it'll probably take a while for me to reach 1000 boards.

 

 

You are at 69%

 

But as phil points out many of the rankings are still 'evolving' (e.g. both mine and yours) as we haven't played 1000 boards.

[WellSpyder]

I think it will take a little while to see how accurate the scheme is, but I suspect it will be of most interest to improvers up to goodish club players. For the better players, I don't think a few stats will make much difference to perceptions formed by playing each other, records in national competitions, and perhaps the Gold Point list that ranks creditable achievements in decent events. The biggest weakness of the scheme, of course, is that it takes no account of teams' results, but only those achieved as a pair, while it is the biggest teams competitions that I suspect most of the top players think define how good their results are.

 

I don't think a low number of boards is likely to be as much of a factor behind surprising results as one might think, since grades are only recorded for those who have played several hundred hands over the past 3(?) years. And incidentally, if you haven't yet reached this I guess you only will if your rate of play is higher than it used to be, since although results have only just been published they do have a full record of results going back for a while. However, we will have to see how volatile the grades appear to be - it is possible to look back over the history of your own grade, and I was a bit alarmed to see that my current grade (60.5%) apparently represents a severe fall off from a grade of 67.5% or something nine months ago. Whatever my partners might suggest after a session that hasn't gone as well as it might have, I certainly don't think I have got that much worse over the past year......

[Finch]

So, today the NGS went live, and you can look up EBU members by name to find out how good they are...apparently.

 

The NGS ranks you according to the MP score you would expect if you were to play in a field of the whole EBU. Supposedly. So I was interested about how accurate we would expect this to be? This is the primary question I would like ppl to answer.

 

 

I think it probably gives (will give, with enough history) a reasonable relative ranking for the vast majority of people whose main bridge is playing club duplicate, and who play at more than one club (or at least where a fair number of the players at their club also play at other clubs). The top and the bottom are likely to be somewhat volatile. If you look at the (current) top 50 or so, you see a mix of the top English players and some other people who play a lot of club bridge, often with some very good partners.

 

If you look at the 'top 20' partnerships, you see the same - a mix of good partnerships and some other pairs who aren't; my guess would be that some of them only play at a few local clubs so aren't yet properly calibrated with the rest of the country.

 

Also, it obviously can't work for individuals if they only play in one partnership: if you only play matchpoints in one partnership, and your partner only plays with you, you will have the same grading which will be the average.

[gnasher]

Everybody's better than me:

Wank: 68.71

DBurn: 63.8

MickyB: 62.26

Bluejak: 61.04

Lamford: 59.22

Vampyr: 58.22

[WellSpyder]

So I was interested about how accurate we would expect this to be? This is the primary question I would like ppl to answer.

 

Everybody's better than me:

Wank: 68.71

DBurn: 63.8

MickyB: 62.26

Bluejak: 61.04

Lamford: 59.22

Vampyr: 58.22

 

Does that answer your question about accuracy already, Phil?

[rogerclee]

Chris Champion runs a similar system in the ACBL, though it is completely unofficial, called the Power Rating.

 

http://www.coloradospringsbridge.com/PR_FILES/PR.HTM

 

Previous discussion: http://www.bridgebase.com/forums/topic/40844-lol-meck-owns-everyone/page__hl__%2Bpower+%2Brating

 

http://www.bridgebase.com/forums/topic/32643-suggestion-for-a-newer-method-for-masterpoint/page__p__375331__hl__%2Bpower+%2Brating__fromsearch__1#entry375331

 

It takes all your matchpoint games from sectional/regional/national tournaments in the last two years (club games also if your clubs submit data) and factors in the strength of field and strength of partner to try to guess how good you are at matchpoints. The result is a bit questionable, there are obviously some players very high up who are not top players, and there are also some top players who are ridiculously underrated (Eric Rodwell at #358, mostly due to Meckstroth being overrated as the undisputed #1). I'm not sure how your NGS stats are computed, but I imagine the principle is similar.

 

In my opinion, PR (and if it's similar, NGS) shouldn't be taken very seriously at the highest rankings, but it is generally (but not always) a fine predictor of skill among people in the good to very good range.

[MickyB]

I'm told that it's just your last 1,000 boards, seems odd to not include results from six months ago for people who have played over 1k boards in that time.

 

Your last 50 boards are worth 10% of your grade.

[mgoetze]

The full description of the NGS shows that a lot of thought has been put into this with a solid mathematical foundation, certainly more than I can discern in the "Power Ratings". Nevertheless, the whole scheme suffers from the absurd design goal of rating individuals rather than only partnerships. This leads to phenomena such as the following: let's say we have two successful partnerships, we'll give them fantasy names such as Bocchi-Duboin and Sementa-Madala. Bocchi and Duboin both have a rating of 75, whereas Sementa and Madala both have a rating of 65. Now, these partnerships dissolve and new partnerships are formed, say Sementa-Duboin and Bocchi-Madala. What happens now is that, no matter how good Madala becomes, even if he eventually turns out to be a better player than Bocchi, Bocchi will always have a rating 10 points higher than Madala as long as they are playing only with one another. The NGS is not as susceptible to this problem as other similar systems, but the basic principle is in effect.

 

So, personally, I expect that the "partnership grades" will be actually useful and the individual grades will not.

[mgoetze]

Your last 50 boards are worth 9.745% of your grade.

 

FYP

[gnasher]

Since the end of 2010, my grade has fluctuated between 47.73 and 60.65.

 

I had one particularly damaging game at the Young Chelsea on 31st July last year, where apparently I scored 0%, causing a drop in my grade from 60.65 to 51.70. I have a vague recollection of winning an IMP pairs on that date; if I'm right about that, I suppose it's true that we scored zero matchpoints.

 

Since then I've clawed my way back to nearly 56.7, by means of another IMP pairs at the YC and a mediocre performance in the EBU Year End tournament. Two matchpoint games at the YC in January appear on my record but didn't affect my grade.

 

Edit: This wasn't meant as a criticism: I appreciate that it's a new system, with masses of data from thousands of different sources, and a lot of scope for human error. It's natural for there to be some problems early on.

Edited March 5, 2012 by gnasher

[gordontd]

I had one particularly damaging game at the Young Chelsea on 31st July last year, where apparently I scored 0%, causing a drop in my grade from 60.65 to 51.70. I have a vague recollection of winning an IMP pairs on that date; if I'm right about that, I suppose it's true that we scored zero matchpoints.

It was a Swiss Pairs with IMP scoring, so I suspect that's what confused the scoring system. I did check the original XML file, and it does show your 103 VPs, so I've forwarded it to the EBU and deleted the session from the club's record. Hopefully it'll all get corrected soon.

 

Your rating aside, I think it looks like a great system and has already had people here fighting to get control of the mouse in the bar, and calling out names of people to look up.

[gordontd]

So, personally, I expect that the "partnership grades" will be actually useful and the individual grades will not.

I think most players (especially the stronger ones) will have enough different partners that this will not be so.

[Tomi2]

we have some comparable scoring in Germany too.

I think it is very hard to find a fair rating system. The one used here has lots of negative points, for example:

I played some of the past mixed pairs championships with my mother, especially when my father was not available, because of playing other tournaments. We always do well, but thats not the point. Last year my parents won a bronze medal, this weekend I played with my mum, but since she plays no other championships their medal effectively costs me kinda 1% of my rating for this weekend. (compared to "if they didnt play at all last year")

additionaly I played open pairs after that event with my father, so I guess, their good mixed result costs me about 30 places...

atm my mother is ranked higher in MP than me and its virtually not possible for me to score so well in open pairs the catch her. my only hope must be, that she plays with my father next time again and that they suck - that would improve my rating...

But my parents both play bridge for 30 years, so ONE result will not make them better/worse players than they are. Their raiting would be influenced by the next tournament by 18% in our German system...

 

the EBU and German rating seems to take the average of both players to dertermine the pairs strength and uses some formula like: (strength of your partnership compared to the field) * your result = the result, that effects your new ranking

maybe it would be some idea, to get two different results for both players in a partnership by counting some weightend average for each player

strenght of partnership for the score of A = (2*A+B) /3

strenght of partnership for the score of B = (A+2*B) /3

 

so if A(60%) and B(50%) form a partnership in a 55% field, they wont need 55% to reach "hold" their rating but:

A will need kinda 56.7% and B will need 53.3% (of course those precentages are not = 55% so the strenght of the field will have some effect)

if they now play a 55% session the ranting of A will drop a bit and the rating of B will rise

[Tomi2]

we have that rating in "Butler" too, so your butler scoring in teams events will count. After playing a weak season with some new partner, the most successfull German player in the past 20 years (in national events that is) Dr. Wladow dropped to place 68!! Now he formed his successfull partnership with Elinescu, who waited in the top 10 of this rating, and in the current league they scored 1.75 imp/board... so they show again, that they are the best players but Elinescu rating will kinda explode because Wladow is soo much underrated. Their difference at the bedinning of the year was 0.5 imp/board, so Elinescu effectively gets 0.25 extra imps per board becaue his partner had a weak season in between, thats WAY TOO MUCH imo!

[awm]

Is there a link to the math behind this system? As a mathematician I'd be interested to look. Most of the systems I've seen (including power ratings and Lehman) have serious flaws.

[MickyB]

http://www.ebu.co.uk/documents/EBU-NGS-Fullguide-V12-final.pdf

[phil_20686]

Is there a link to the math behind this system? As a mathematician I'd be interested to look. Most of the systems I've seen (including power ratings and Lehman) have serious flaws.

 

If you put ebu into google you will find the ebu website, and from there you can follow the links to all the documentation. It is mostly still on the main page as its pretty new.

 

I do think its pretty rare for top players to have a large number of MP boards with the same partners. E.g. serious partners mostly do not play much MP except for the occasional `spin' normally involving drinking. Its doubly hard for me as fully half of what I play is in Scotland, which is not a part of the scheme. Half of the boards I have played that count are with random friends who I play with seldom or with friends of my brother who I play with in a `mentor' type capacity. I would imagine this is a problem for the partnership rankings when they do not take the IMP competitions into account, as like it or not those are where most of the serious pairs turn up in a big way.

 

I think it will take a bit more time for the rankings of insular clubs to stabilise. I wonder if the ranking scheme was applied only to national tournaments would the results change greatly, if yes, that would seem to indicate that it does not properly take account of the strength of various fields, but this could be an intrinsic problem. In the same way that an easy maths test does not appropriately rank the top end, it could be that a good player in a poor field simply cannot to well enough to overcome the discount from playing in a poor field. Another way of saying that is that I might average 60% in an average field, and still average 60% in a much poorer field, as there is less an less gain from being better as results become more random. This would represent an intrinsic limitation on the system, and people who win in better fields would do better than the same players winning in a poorer field.

[phil_20686]

Does that answer your question about accuracy already, Phil?

 

I dont think so, as few of us have enough boards that "count", especially those of us who mostly play teams events rather than MP.

[Vampyr]

I think it will take a bit more time for the rankings of insular clubs to stabilise.

 

I think it will take forever. For clubs with few points of contact with the wider EBU, the occasional point of contact will cause wild fluctuations one way or another. Suppose a top pair pop into the a little club somewhere in Cumbria, whose members rarely, if ever, play outside that club. If the top pair have an off night and score poorly, this little club will be deemed a very strong field and the members' ratings will be inflated in perpetuity. Obviously the opposite effect is also possible.

 

I realise that this is an oversimplification, but I think that the effect is real and will be long-lasting. I wrote at length to John Carter and, after his death, to Sally Bugden about a number of concerns I had about the system. Other important (to me anyway) ones were rating individuals instead of partnerships, and the bell-shaped curve -- I think that the people rated "2" and "3" should be in larger groups so they don't feel stigmatised. Perhaps the ratings should even be concealed until a person reaches a certain level.

 

But mainly I worry that this scheme, were it to be taken seriously, would discourage casual and one-time partnerships, mentoring and the like, and would discourage people from showing up without a partner to clubs that have hosts or guaranteed partners. If/when team games begin to be included, people might not wish to team up with friends, in order to protect their ratings -- and for many, this will mean not participating at all. (This might be different if the Butler scores and not the team results were counted, of course.)

 

So in general, my thinking is that if people start to care about their ratings, I think it would have negative effects on bridge in England. I hope that I am wrong.

[gnasher]

Calculating event grading values for teams of four events

 

There are no plans to include within the NGS results from head-to-head teams-of-four matches, where boards are played at just two tables, as the scheme would be unable to differentiate the relative performances of the two partnerships within the team.

That's a strange argument. Exactly the same problem exists with differentiating the members of a partnership. In a pairs event they deal with this by adjusting your result by the difference between your grade and your partner's; in a teams event an equally good (or bad) solution is to adjust by the difference between your grade and your teammates' average grade.

 

It is intended, though, to include results from most other types of teams events as it becomes possible to analyse teams games as IMPed Pairs. This is because we are then able to reduce significantly the impact of the actions of our team mates from the calculation of our gradings.

 

Such analysis will become possible for Multiple Teams-of four events and Swiss Teams events provided that Butler IMP scores can be obtained from the scoring program. For Teams-of-eight matches, which are common in County Leagues, we would need to analyse the Butler IMP scores, and indeed, for a long time, Butler IMP scores have commonly been calculated for matches between large teams to assess relative performance by the various pairs within a team.

That addresses one of Vampyr's concerns, but I'm not terribly keen on it.

 

In a Swiss Teams event, Butlers (or cross-IMPs) simply won't produce the right answers, because the results are strongly dependent on the strength of the team you are playing.

 

In an event with seating rights, Butler or cross-IMP scores are also of questionable value, because the stronger pairs tend to pick each other. And it creates a perverse incentive to pick the weaker opposing pair so as to inflate your grade.

 

The only situations where I can see this working are round-robin events and leagues, where either you don't pick your opponents or you play against both opposing pairs.

[gnasher]

Two matchpoint games at the YC in January appear on my record but didn't affect my grade.

Those two games were played with a visitor from overseas who had never played in England before. From the NGS Guide: "Only pairs where both players can be identified are included in the NGS processing of a session."

[Cyberyeti]

I'm not even on there as I rarely play pairs. My partner is in there at 62.10.

[helene_t]

haha I have the same ranking as Andy :) How is this possible? I get on average some 50% on club nights