Hand dealt vs. computer generated

[The_Badger]

I rarely play at the bridge club now because of illness and disability, but I'll be interested if other older members still remember when bridge hands were less exciting generally. (And that's a personal view.)

 

As bridge hands were manually dealt, there seemed to me less distributional hands, less competitive bidding, less seat of the pants decision-making.

 

I say this, as I remember when computer-generated hands first appeared at the English Bridge Union summer congress in Brighton in the late 1970s/early 1980s. There were quite a few complaints generally about how 'wild' they were - in effect, a lot more distributional than previously.

 

I presume most bridge hands, except for rubber bridge, are now computer-generated, and I wonder if other players have noticed this difference: hand dealt vs. computer-generated, or am I imagining this difference?

[Stefan_O]

I rarely play at the bridge club now because of illness and disability, but I'll be interested if other older members still remember when bridge hands were less exciting generally. (And that's a personal view.)

 

As bridge hands were manually dealt, there seemed to me less distributional hands, less competitive bidding, less seat of the pants decision-making.

 

I say this, as I remember when computer-generated hands first appeared at the English Bridge Union summer congress in Brighton in the late 1970s/early 1980s. There were quite a few complaints generally about how 'wild' they were - in effect, a lot more distributional than previously.

 

I presume most bridge hands, except for rubber bridge, are now computer-generated, and I wonder if other players have noticed this difference: hand dealt vs. computer-generated, or am I imagining this difference?

 

Controversial topic ;)

 

No, I can not say I have felt that deals on BBO, for example, are particularly distributional, more than expected.

 

Also, writing a good random shuffling-algorithm is a very simple task, so I doubt that computer-deals,

generally, would be more distributional than predicted by the statistical odds.

Of course, it happens from time to time that dumb programmers do the most stupid things one would never imagine,

so I cannot guarantee every bridge-program in the world generates deals as predicted by the statistical odds :)

 

If your perception really has any basis in actual facts, however, there is one old theory going around:

 

When people hand-shuffle a deck, they are lazy and do not shuffle the cards enough.

If this leads to that cards of the same suit remain together in the deck more frequently,

than they would after a true random-shuffle,

the result would actually be that hand-dealt deals would be LESS distributional than they should be according to statistical odds.

[barmar]

No, I can not say I have felt that deals on BBO, for example, are particularly distributional, more than expected.

Outside of BBO, I think most computer-dealt hands use programs that incorporate the BigDeal dealing program, which is generally considered the gold standard, so they're probably about as good as is reasonably possible. We also had its author perform a statistical analysis of BBO deals, and he couldn't find any glaring anomalies, although our dealing algorithm is much simpler than BigDeal.

[Phil]

When others have looked at this topic, imperfect shuffles by players actually generate wilder hands than computer generated hands so the whole

'computer-hands' thing is a myth generated by club players.

[neilkaz]

When others have looked at this topic, imperfect shuffles by players actually generate wilder hands than computer generated hands so the whole

'computer-hands' thing is a myth generated by club players.

 

+1 and agreed 100%

[The_Badger]

Controversial topic ;)

 

When people hand-shuffle a deck, they are lazy and do not shuffle the cards enough.

If this leads to that cards of the same suit remain together in the deck more frequently,

than they would after a true random-shuffle,

the result would actually be that hand-dealt deals would be LESS distributional than they should be according to statistical odds.

 

Thanks for that Stefan_O

 

So there is a logical explanation after all.

 

Barmar said: Outside of BBO, I think most computer-dealt hands use programs that incorporate the BigDeal dealing program, which is generally considered the gold standard, so they're probably about as good as is reasonably possible. We also had its author perform a statistical analysis of BBO deals, and he couldn't find any glaring anomalies, although our dealing algorithm is much simpler than BigDeal.

 

So to check this myself, I analysed 100 random deals on BBO. Using the same player. The probability of holding either a 6 card or a 7 card suit in your hand works out about 20%.

 

And guess what, even though certain blocks of deals were more distributional, veering towards and over 30%, over the 100 deals the probability came out at, guess what, about 20% :)

 

Yes, it is a small sample for analysis but I now realise that computer-generated deals are statistically sound. Thanks for all your comments and observations.

[Stefan_O]

When others have looked at this topic, imperfect shuffles by players actually generate wilder hands than computer generated hands ...

 

Seems strange to me... as long as you deal 1-1-1-... (and not "goulash" 5-4-4 or similar), how can that generate wilder hands?

 

Any links to such research/analysis?

[helene_t]

In rubber bridge I believe that handshuffles makes flat hands because cards of the same suit tend to be in the same trick but in duplicate I would think the difference would be tiny.

[hrothgar]

FWIW, I spent far to much time looking into the intricacies of the system that the ACBL and the USBF use to generate hands.

 

Executive summary: The system that the ACBL uses to generate hands is both plausibly insecure and statistically problematic.

 

When I say "plausibly insecure" I mean that someone could invest a relatively small amount of money, crack the ACBL's hand generator, and use the first hand of a tournament to predict all subsequent hands in real time.

 

When I say "statically flawed" I mean that the linear-congruential generator that the ACBL uses to generate pseudo random numbers only covers a relatively small portion of the space of hand deals and the process that the generator uses to walk through this space use a very simple process. If the phase of the hand generator program were to align with the phase of the the lcg all sorts of bad things would happen.

 

At the most basic level, and lcg is not cryptographically secure and should not be used for any kind of statistical simulation.

 

You can find a more detailed discussions at

 

http://bridgewinners.com/article/view/acbl-and-usbf-hand-records-are-plausibly-insecure/#new_1

http://www.metzdowd.com/pipermail/cryptography/2016-August/029965.html

 

I've informed both the ACBL and the USBF about these results and made the suggestion that they either swap out the prng for something real or, alternatively, just standardize on Big Deal instead. I know that Jan Martel is going to talk to some folks at the WBF about Big Deal. The ACBL had originally disputed that there is a problem and has now gone silent on the topic.

[Stefan_O]

In rubber bridge I believe that handshuffles makes flat hands because cards of the same suit tend to be in the same trick but in duplicate I would think the difference would be tiny.

 

Nyah... also in duplicate (hand-dealt) deals, you have the similar situation where declarer plays several rounds of trumps in a row, or just plays off any suit from the top, etc, which happens almost every deal.

This will result in several cards of the same suit laying adjacent when they stuff it back into the card-slots.

 

Next tourney, somebody takes the hands out, just puts them in a pile on top of each other, shuffles too poorly, and many may still remain adjacent,

which skews the suits to be more evenly spread out among the four hands, than after a sound random-shuffle.

[Stefan_O]

The ACBL had originally disputed that there is a problem and has now gone silent on the topic.

 

How did they dispute it?

Is that response available online?

[hrothgar]

How did they dispute it?

Is that response available online?

 

Here is a copy of an email that I received from an representative of the ACBL

 

Hello, Everyone,

 

I was following this conversation on the Bridgewinners website with some degree of amusement, as the degree of code cracking necessary to accomplish a read of the hand record set within a three hour period (a typical play session) does not currently exist. Here is the description of the program used by ACBL, provided by the programmer, Jim Lopushinsky. This was published in the September 2011 Bulletin.

 

"The following is an excerpt from a letter to the editor, with a response by Lopushinsky, published in the September 2011 issue of the ACBL Bridge Bulletin that may help shed some light on the dealing program used by the ACBL.

 

Bill Clough of Lynchburg VA wrote: “The question is whether the ACBL hands are random. Let’s look at some numbers. From the ACBL’s own web site, we can find that a huge number of deals can be generated as the random seed is 2^47, that is 140,737,488,355,328 or 1.4x10^14 possible deals. The seed is equal to the largest possible number of unique deals that can be generated.

“It is true that the total number of possible deals is closer to 2^96, but there are actually 53,644,737,765,488,792,839,237,440,000 or 5.3644x10^28 possible bridge deals.

“Surfing the web a few years back, I found an article that helped to understand these enormous numbers. If you have one atom of gold for each possible bridge deal, a gold cube could be formed 3.9 feet on a side and weigh 19 tons with a value more than $800 million.

“Doing the same for the ACBL deals, the cube of gold formed would be 1/1500 of an inch and weigh .05 microgram – less then the ink of the dot on an “i” – with a value of 1/5000 of a cent.

“So are the ACBL deals random? Yes, of course, they are random – as random as the generating program can make them.”

 

This was Lopushinsky’s response:

“The writer is correct as to the number of hands that can be generated from one seed, but the seed is arbitrarily assigned for each set of hands.

“The random number generator uses the linear congruential algorithm and 48-bit integer arithmetic. This will generate 2 to the 47th power different numbers before repeating without any outside influence (140,737,488,355,328 numbers).

“Outside influence occurs in the form of manually dealt hands, starting seed numbers, time of day, etc., to make the number of numbers virtually infinite, and to guarantee that the same hand will not be repeated. Ninety-six bits take part in the operation with the high 48 bits acting as the overflow.

“The operation works as follows: The hand record set number is used as the starting seed. This seed is multiplied by day of the month, current minute, current hour, current second, current day of week and current hundredth of second. This number is then multiplied by a large prime number (5DEECE66D in hexadecimal). Thirteen is then added to this. The lower 48 bits is then saved and used as the seed to generate the next random number. The overflow (48 high bits) is then doubled and multiplied by the range requested (1 - 52) and the overflow from this is used as the random number.”

 

The computer used to run the ACBL hand record generator is a stand alone desktop computer that is not connected to any network. Since Jim's retirement in May, I have been responsible for maintaining our stock of electronic hand records used by our Tournament Directors.

 

When we need additional hand records, we generate two thousand sets at a time (72,000 deals). The seed deal is manually entered at the beginning of the process, and is not part of any set subsequently produced. Sets are never reused, and the number of the set is not released publicly until after play. The date and time when the hand record was prepared (we keep a five thousand set buffer) does not leave ACBL Headquarters.

 

With so many of the variables involved in the hand record set creation process kept protected and secure, I doubt anyone will be cracking the security in sufficient time to make use of the data they develop.

 

Regards,

 

Keith Wells

ACBL Tournament Technical Analyst

 

 

6575 Windchase Blvd.

Horn Lake MS 38637-1523

ph: 662-253-3165

w: www.acbl.org

eml: keith.wells@acbl.org

[Stefan_O]

Here is a copy of an email that I received from an representative of the ACBL

 

Thanks hrothgar,

 

Interesting :)

 

Yes, making sure the algorithm cannot feasibly be "cracked"/reverse-engineered for foul purposes,

is certainly a much harder task than generating deals with expected random/percentage distributions.

 

Looks like a totally homebrewn approach, which is always suspect in crypto-contexts.

At least, I dont see they make any attempt at all to give a cryptographicly sound argument/proof/reference, why this would be a secure one from a reverse-engineering viewpoint.

 

"With so many of the variables involved in the hand record set creation process kept protected and secure, I doubt anyone will be cracking the security in sufficient time to make use of the data they develop."

 

Incidently, the same casual attitude the Nazi's maintained until the end -- years after the Enigma was cracked.

"We are just so sure nobody will ever figure this one out!"

:)

[hrothgar]

 

Looks like a totally homebrewn approach, which is always suspect in crypto-contexts.

 

 

The choice of the constant 5DEECE66D suggests that the function might share a common ancestry with

 

Java's java.util.Random

POSIX [ln]rand48

glibc [ln]rand48[_r]

[Stefan_O]

The choice of the constant 5DEECE66D suggests that the function might share a common ancestry with

 

Java's java.util.Random

POSIX [ln]rand48

glibc [ln]rand48[_r]

 

Mmmm... right... and which is listed among non secure LCGs on wikipedia...

 

"They also must not be used for cryptographic applications; see cryptographically secure pseudo-random number generator for more suitable generators."

 

https://en.wikipedia.org/wiki/Linear_congruential_generator

[pigpenz]

A lot of Barry Crane's rules for playing suits was based on old fashioned shuffling from what I have read

[mgoetze]

When others have looked at this topic, imperfect shuffles by players actually generate wilder hands than computer generated hands so the whole

'computer-hands' thing is a myth generated by club players.

I'd love a citation for this, in my admittedly small sample of playing poorly-hand-shuffled boards the distributions have certainly seemed flatter to me.

[1eyedjack]

It would come as no surprise to me if there were differences between any noticeable patterns of deviation from random observed by imperfect shuffling of (1) packs habitually used in duplicate sessions, contrasted with (2) packs habitually used in rubber. It may become increasingly hard to measure this with the tide of computer dealing taking over duplicate clubs. Not that anyone would spare the time and effort to do any serious research.

 

I wonder also if it is relevant to take into account that a new pack of cards typically comes sorted by suits, while an old pack of (perhaps well shuffled) cards with evident wear gets discarded.

[661_Pete]

I've wondered about this at times.

 

It's a rather subjective thing, isn't it: thinking you've been dealt more than the 'average' number of 7-card or 8-card suits in a session. Because one remembers such deals, and forgets the flat deals which happened more often?

 

I have noticed that a lot of the BBO-hosted tourneys are described as 'goulash'. Haven't tried one myself, but if they mean what I think they mean, does it imply that the BBO dealer has a 'distributional' setting that can be switched on or off depending on the type of tourney? If so, is there a possibility that this switch gets inadvertantly turned on for a deal in one of the casual bridge clubs? Just asking....

 

When people hand-shuffle a deck, they are lazy and do not shuffle the cards enough.

This raises an interesting point of etiquette. When playing with real cards, is it proper and correct for the dealer's partner to continue shuffling for the entire time that the dealer is dealing out the hand? I must admit, I sometimes finish shuffling when my partner is still only half-way through dealing. Is this wrong? Am I one of the 'lazy' ones? :unsure:

 

Also, at our local U3A club (where all shuffling is by hand, I assure you - and the OP is very welcome if he's in our area), we play mostly Chicago scoring, but we still keep the cards of played tricks in front of us, duplicate-style. We don't gather them into tricks, rubber-style. Possibly this makes a difference?

[barmar]

I have noticed that a lot of the BBO-hosted tourneys are described as 'goulash'. Haven't tried one myself, but if they mean what I think they mean, does it imply that the BBO dealer has a 'distributional' setting that can be switched on or off depending on the type of tourney? If so, is there a possibility that this switch gets inadvertantly turned on for a deal in one of the casual bridge clubs? Just asking....

These aren't dealt by the normal BBO dealer. They generate the hands themselves, and upload them into the tournament.

 

You can use a program like Dealer to specify hand distribution constraints.

[barmar]

When I say "plausibly insecure" I mean that someone could invest a relatively small amount of money, crack the ACBL's hand generator, and use the first hand of a tournament to predict all subsequent hands in real time.

You mean the first day of an NABC they can predict all the hands for the rest of the week? Wouldn't that require them to use the same run for all sessions during the tournament, and in a predictable order?

 

Or just that from first board of a session they can predict the remaining hands of the session? But unless they can get the first hand ahead of time, does that do them much good? And if they can get the first hand record, they could probably get the hands for the entire session. I suppose someone could play board 1, go to the restroom, enter the board in their phone and get the rest of them -- is that the concern? In any "important" event (like the late rounds of big events like Spingold), players aren't allowed to leave the playing area unsupervised.

[661_Pete]

I'm wondering, what's the freakiest hand ever seen on BBO (outside of the goulash's, that is)? I think the longest suit I've ever had is a 9-carder - I have a vague recollection of an 11-1-1-0 in live bridge many years ago (missing the AQ of my long suit, I couldn't even make game.....) but it's all rather hazy.

A few weeks ago on BBO, my partner picked up a 7-6-0-0 which is quite a rarity I think.

[Stefan_O]

I'm wondering, what's the freakiest hand ever seen on BBO (outside of the goulash's, that is)? I think the longest suit I've ever had is a 9-carder - I have a vague recollection of an 11-1-1-0 in live bridge many years ago (missing the AQ of my long suit, I couldn't even make game.....) but it's all rather hazy.

A few weeks ago on BBO, my partner picked up a 7-6-0-0 which is quite a rarity I think.

 

 

I had 7-6 around the 20th of June this year:

 

http://tinyurl.com/z36qqum

.

.

.

.

.

.

Only a few days later, the robot got its revenge :D :D (I havent seen anything like it, since)

 

http://i.imgur.com/Zhc3JpS.png

[hrothgar]

Here's an amusing hand from yesterday's BBF event

 

http://www.bridgebase.com/tools/handviewer.html?bbo=y&lin=pn|kalatavg12,hrothgar,jimcloh,barmar|st%7C%7Cmd%7C1S39QH9JQD28JC57JQ%2CS5KH367D34567QKAC%2CS47TH8TKD9TC469KA%2C%7Crh%7C%7Cah%7CBoard%2015%7Csv%7Cn%7Cmb%7Cp%7Cmb%7C2C%7Cmb%7Cd%7Cmb%7Cp%7Cmb%7C2N%7Cmb%7C3D%7Cmb%7Cp%7Cmb%7C3S%7Cmb%7Cp%7Cmb%7C4D%7Cmb%7Cp%7Cmb%7C4H%7Cmb%7Cp%7Cmb%7C5D%7Cmb%7Cp%7Cmb%7Cp%7Cmb%7Cp%7Cpc%7CCA%7Cpc%7CC2%7Cpc%7CC5%7Cpc%7CD3%7Cpc%7CDA%7Cpc%7CD9%7Cpc%7CC3%7Cpc%7CD2%7Cpc%7CDK%7Cpc%7CDT%7Cpc%7CC8%7Cpc%7CD8%7Cpc%7CDQ%7Cpc%7CC4%7Cpc%7CCT%7Cpc%7CDJ%7Cpc%7CD7%7Cpc%7CC6%7Cpc%7CH2%7Cpc%7CCJ%7Cpc%7CD6%7Cpc%7CC9%7Cpc%7CH4%7Cpc%7CCQ%7Cpc%7CD5%7Cpc%7CCK%7Cpc%7CH5%7Cpc%7CC7%7Cpc%7CSK%7Cpc%7CS4%7Cpc%7CS2%7Cpc%7CS3%7Cpc%7CS5%7Cpc%7CS7%7Cpc%7CS8%7Cpc%7CS9%7Cpc%7CHQ%7Cpc%7CH3%7Cpc%7CH8%7Cpc%7CHA%7Cpc%7CSA%7Cpc%7CSQ%7Cpc%7CH6%7Cpc%7CST%7Cmc%7C12%7C

 

(Still somewhat surprised Barry didn't raise to 6♦)

[661_Pete]

Only a few days later, the robot got its revenge :D :D (I havent seen anything like it, since)

I think you can blame your robotic partner there, for not leading their ("its"?) singleton spade and at least hold opponents to ten tricks. Especially at MPs scoring.

 

I've never tried playing with robots on BBO and I don't have any intention to. Robotic dealers I can live with, but robotic players...... :blink:

http://suptg.thisisnotatrueending.com/archive/20534202/images/1346375601826.jpg