Bidding Question Analysis Program

[RossSCann]

A couple of months ago I posted the bidding question below from Frank Stewart's daily column

I am modifying my bridge program to analyze alternative bids for 500 random deals;

At the time it seemed to me (low intermediate player) that the best bid would be to pass

I ran an analysis that confirmed my thought. A number of readers protested, some violently,

saying everyone knows you bid 2 H's

Well, there was a bug in my program.

After various distractions I have gotten back to the issue, with the following new results.

2 H's wins by a small amount at MP's.

 

[hv=pc=n&s=sjt652hqjt9d7ck42&d=n&v=0&b=1&a=1dp1sp1np]133|200[/hv]

Alternative rebids by South are 2 H's or Pass

 

Total Score Top Boards

Key bid 1 2H 2228 240

Key bid 2 Pass 3846 228

 

Flat boards 32

 

 

Distribution of High Card Points For North - Number of Hands

11 12 13 14 15 16 17

0 128 498 274 78 22

 

If anyone wants to look at any of the 500 random hands I can email a file.

I am interested in the topic of analyzing alternative bids by generating a large number of random hands to see the result.

 

Results for 500 good cases for another run w/ more detail

 

Dealer N opens 1 D and S responds 1 S, N now bids 1 Nt: Key bids are 2 H's or Pass

            Total Score  Top Boards
Key bid 1   2H   2860        249
Key bid 2 Pass   3900        217

Flat boards 34 

Distribution of High Card Points For North - Number of Hands
 11  12  13  14  15  16  17
  0 122 514 269  74  23

Distribution of Imps Won - Number of Boards
Imps       2H    Pass
 1      98      25
 2      54       7
 3      14       8
 4       0       0
 5      33      39
 6      17      35
 7      14      26
 8       8      14
 9       4       6
10       4       3
11       2       1
12       1       0
13       0       0
14       0       0

Total   787     857

Contracts Reached After Bid Choice One 2H
Declarer  Bid   Suit   Made   Cases
  South  2  Heart       2    91
  South  2  Spade       2    84
  South  2  Spade      -1    62
  South  2  Heart       3    60
  South  2  Spade       3    53
  South  2  Heart      -1    51
  South  2  Spade      -2    31
  South  2  Heart      -2    21
  South  2  Heart       4    13
  South  2  Spade      -3    10
  South  2  Spade       4     7
  South  2  Heart       5     4
  South  2  Heart      -3     3
  South  2  Heart      -4     2
  North  3  Notrump     3     2
  North  3  Notrump    -1     1
  South  2  Spade      -5     1
  North  3  Notrump     5     1
  North  2  Notrump     3     1
  North  2  Notrump    -1     1
  North  2  Notrump     2     1

Contracts Reached After Bid Choice Two Pass
Declarer  Bid   Suit   Made   Cases
  North  1  Notrump     1   179
  North  1  Notrump     2   109
  North  1  Notrump    -1    94
  North  1  Notrump     3    55
  North  1  Notrump    -2    36
  North  1  Notrump     4    13
  North  1  Notrump    -3     9
  North  1  Notrump    -4     2
  North  1  Notrump     5     2
  North  1  Notrump    -6     1

Distribution of North Hands

Number of
Cases Spades Hearts Diamonds Clubs
  85     3      3       4      3
  71     2      3       5      3
  64     2      4       4      3
  43     2      3       4      4
  40     2      4       5      2
  31     3      3       5      2
  26     3      2       5      3
  24     3      2       4      4
  23     3      4       4      2
  22     1      4       4      4
  14     2      2       5      4
  13     2      2       4      5
  10     4      3       4      2
   8     1      4       5      3
   6     4      4       3      2
   5     1      3       5      4
   5     4      2       4      3
   4     1      3       4      5
   4     4      2       5      2
   2     1      2       5      5

Edited October 17, 2012 by RossSCann
Replaced hand/bidding description with diagram

[bluecalm]

I like it. Some time ago I was thinking about writing some scripts to make it automatic but then I got distracted by the idea of building "single dummy estimator" which I presumed would be much faster and thus allow for full search instead of generating sample of hands.

Your approach is probably good enough to give many interesting results but there is this stuff about NT contracts making relatively more often than suit contracts comparing to dd results so and I don't know how to correct this bias.

Maybe building some kind of first lead algorithm would help with that (dd results after 1st lead is made are much better estimation of what happens single dummy than the ones before 1st lead).

[Stephen Tu]

Results for 500 random cases

 

Total Score Top Boards

Key bid 1 2H 2228 240

Key bid 2 Pass 3846 228

 

Flat boards 32

I would have expected 2h to win by a larger margin from my previous sim in the previous thread.

 

Distribution of High Card Points For North - Number of Hands

11 12 13 14 15 16 17

0 128 498 274 78 22

 

This still seems odd. Why would North ever have 15/16? 1nt rebid is supposed to be 12-14 in theory. I suppose some awkward 1453s with small spade might decide to downgrade a 15 count, rebid 1nt, which is fine, but how is your script deciding this? Also, why are 12 counts so rare compared to 13 counts? Normally, one expects 12 counts to be more common than 13s by a significant margin. Is your program some ultra-sound bidder, discarding certain 12 counts for some reason?

 

If you posted a histogram of opener's possible shapes and the final contract reached, # of tricks taken, perhaps the errors can be spotted.

 

You could also post your file to some cloud service like skydrive or google drive, in a publicly accessible folder, and post a link, rather than doing it by email.

[aguahombre]

All I know for sure is that as long as partner knows 2H is either to play it there if she has 4 of them, or with 2/3 spades she bids 2S ---and no other bids are possible...I will continue rebidding 2H with those hands and not give it any more thought.

 

Folks who rebid 1NT with a singleton and/or other than 12-14 might have different table results and different considerations. For us, life is simple. Maybe the sims and analysis included some non-partnership continuations after 2H. If so, the bottom line results for the 2H rebid would definitely be worse because of the CHO factor.

[RossSCann]

I would have expected 2h to win by a larger margin from my previous sim in the previous thread.

 

 

 

This still seems odd. Why would North ever have 15/16? 1nt rebid is supposed to be 12-14 in theory. I suppose some awkward 1453s with small spade might decide to downgrade a 15 count, rebid 1nt, which is fine, but how is your script deciding this? Also, why are 12 counts so rare compared to 13 counts? Normally, one expects 12 counts to be more common than 13s by a significant margin. Is your program some ultra-sound bidder, discarding certain 12 counts for some reason?

 

If you posted a histogram of opener's possible shapes and the final contract reached, # of tricks taken, perhaps the errors can be spotted.

 

You could also post your file to some cloud service like skydrive or google drive, in a publicly accessible folder, and post a link, rather than doing it by email.

 

Good suggestions. I didn't know about the free cloud posting sites. I do have all the data you suggest re: distribution and tricks taken, etc.

The bidding is set to that which Frank Stewart uses for his colum i.e. simple. For example 1 NT opening 16-18 HCPts.

This is a work in progress so I am anxious for constructive criticism.

[RossSCann]

All I know for sure is that as long as partner knows 2H is either to play it there if she has 4 of them, or with 2/3 spades she bids 2S ---and no other bids are possible...I will continue rebidding 2H with those hands and not give it any more thought.

 

Folks who rebid 1NT with a singleton and/or other than 12-14 might have different table results and different considerations. For us, life is simple. Maybe the sims and analysis included some non-partnership continuations after 2H. If so, the bottom line results for the 2H rebid would definitely be worse because of the CHO factor.

 

All the final contracts after the 2H rebid by responder are 2H's or 2S's. Obviously after pass all the contracts are 1 NT.

I will post details of all the results per the cloud suggestion above. I hope to develop my program into a useful tool for analysis if I can, so contructive review will be very helpful.

As far as I can tell no such tool now exists with easy user input of the bidding question to be analyzed.

One big question I have is whether to just use the play and defense logic in my program, or get something like "Deep Finesse," which considers the hidden cards.

[fromageGB]

Folks who rebid 1NT with a singleton and/or other than 12-14 might have different table results and different considerations. For us, life is simple.

For us it is an even simpler 2♥ rebid. Playing a "long suit or shortage elsewhere" 1♦, 1NT shows a shortage in spades and guarantees heart support, either 4 card (commonly) or 3 (less often), the latter being sufficient with the shortage.

 

But back on topic, I like the idea of this program and look forward to its development. Unfortunately it is too far out of my line for me to make any useful contribution. However, I like bluecalm's idea of developing an open lead algorithm to feed into the analysis. It would need to be a weighted restriction on the analysis, such as 100% ♣K if leader has ♣KQJ, but more commonly along the lines of 50% low ♥, 30% ♠K, 20% low diamond. Might take a long time for 500 hands! I hate the deep finesse (or equivalent) double dummy results you see on hand printouts, as many times they are ludicrous in practice.

[RossSCann]

For us it is an even simpler 2♥ rebid. Playing a "long suit or shortage elsewhere" 1♦, 1NT shows a shortage in spades and guarantees heart support, either 4 card (commonly) or 3 (less often), the latter being sufficient with the shortage.

 

But back on topic, I like the idea of this program and look forward to its development. Unfortunately it is too far out of my line for me to make any useful contribution. However, I like bluecalm's idea of developing an open lead algorithm to feed into the analysis. It would need to be a weighted restriction on the analysis, such as 100% ♣K if leader has ♣KQJ, but more commonly along the lines of 50% low ♥, 30% ♠K, 20% low diamond. Might take a long time for 500 hands! I hate the deep finesse (or equivalent) double dummy results you see on hand printouts, as many times they are ludicrous in practice.

 

As everyone agrees opening leads are hard. My logic has around 500 different leads based on the bidding and West's hand (S is decl).

I got almost all of them from books on defense, Root, Kantar, etc. I would guess that is 20% of what would be required to equal the leads made by an expert. However, as noted above, double-dummy opening leads would not make sense in determining how a given hand plays out. On the other hand my 500 probably covers more than 80% of all random deals. I would love it if some experts would go through the random deals my program generates and see which opening leads they disagree with.

Also, my posting of details is coming out very messy because the fromating is getting garbled. I have to learn how to maintain the formating when adding to this forum.

[Zelandakh]

On the other hand my 500 [hands] probably covers more than 80% of all random deals.

This looks like one of the sig quotes of the year!

[aguahombre]

On the other hand my 500 probably covers more than 80% of all random deals.

This looks like one of the sig quotes of the year!

Yogi Berra: "90% of this game is half-mental."

[PhilKing]

I would have analysed 625 hands, thereby solving bridge.

[antonylee]

Using redeal (https://github.com/anntzer/redeal), linked against Bo Haglund's DDS:

 

from redeal import *

predeal = {"S": H("JT652 QJT9 7 K42")}

mp = [0, 0, 0]
imp = [0]

def accept(deal):
   n = deal.north
   # OP plays 16-18 NT opening, so be it.  I assume no 3-card raise possible.
   return (balanced(n) and 12 <= n.hcp <= 15 and len(n.spades) < 4 and
           (len(n.diamonds) > len(n.clubs) or len(n.diamonds) == 4))

def do(deal):
   n = deal.north
   score_pass = deal.score("1NN") # assumes non-vul
   if len(n.spades) + 1 > len(n.hearts):
       score_bid = deal.score("2SS")
   elif len(n.spades) + 1 < len(n.hearts):
       score_bid = deal.score("2HS")
   else:
   # assume we always end up in the worst contract between 2H and 2S if equal length
       score_bid = min(deal.score("2SS"), deal.score("2HS"))
   if score_bid == score_pass:
       mp[1] += 1
   elif score_bid > score_pass:
       mp[2] += 1
   else:
       mp[0] += 1
   imp[0] += imps(score_bid, score_pass)

def final(_):
   print(mp)
   print(imp)

 

I get, for 1000 hands:

[189, 100, 711]
[1557]

 

i.e. +1.6 imp per board for, or +0.5 board per board (mps) for bidding 2H instead of 1N, at worst.

 

The difference is so huge that I don't think using single-dummy simulations will change anything qualitatively.

 

(PS: There is some issue with the GUI version, only the console version works now. Also I still have to upload the code to make it work on Windows... Unix-systems only right now.)

[Stephen Tu]

As far as I can tell no such tool now exists with easy user input of the bidding question to be analyzed.

One big question I have is whether to just use the play and defense logic in my program, or get something like "Deep Finesse," which considers the hidden cards.

 

Define "easy user input". The thing is, most of us who are interested in simulation are computer-savvy enough to deal with the simple programming scripts needed to operate dealer programs such as Thomas Andrews' deal program or van Stavarens dealer. Perhaps there is some interest in making a GUI front-end to these for people who can't deal with programming scripts? Or maybe some libraries of scripts/functions for those tools that have decent constraints for possible std bidding actions, opening bids/preempts, overcalls, competitive doubles? But otherwise to some extent you are just reinventing the wheel, if you are just going to set up something which deals hands with constraints and feeds it to a double-dummy solver.

 

The problem with relying on the bid/play/defense logic of your program, IMO, is that from what I have seen (from downloading your trial program), the logic can be pretty awful. Not at a level where I'm particularly interested in the results. Why would I care what works best against a bad computer player? I'd rather see someone do the effort to be able to feed generated hands and a specified auction into GIB's bridge.exe, and obtain single-dummy results, however slow that might be. This would especially be useful for game and slam hands, where DD analysis has flaws such as always counting suits such as AJx vs. KTx as zero losers, and picking up both onside Qs and offside doubleton queens on combos like AKJxx vs. xxx.

 

If you want to maintain whitespace formatting in pasted text, encase it in [ code] [ /code] tags without the space in the tag, compose the text with a monospace font.

[antonylee]

I agree with bluecalm that single dummy leads would be a big step towards more realistic results. I remember seeing some stats posted by bluecalm according to which the opening lead, in world-class play, easily blows up ~0.5-1 imp on average against the defense, whereas the rest of the play is much closer to DD. Also, calculating realistic opening leads, either rule-based, or simulation-based, seems reasonably doable to me.

 

PS: Right now redeal does not support computing DD results for deals in which some cards have been played; this is mostly because I am still looking for the right syntax to input such cases. I'd be interested in your suggestions.

 

PPS: The GUI is fixed now... still have to work on a proper Windows installer. If you already have Python installed however (or use Linux/Mac), again, feedback is welcome :)

[RossSCann]

Define "easy user input". The thing is, most of us who are interested in simulation are computer-savvy enough to deal with the simple programming scripts needed to operate dealer programs such as Thomas Andrews' deal program or van Stavarens dealer. Perhaps there is some interest in making a GUI front-end to these for people who can't deal with programming scripts? Or maybe some libraries of scripts/functions for those tools that have decent constraints for possible std bidding actions, opening bids/preempts, overcalls, competitive doubles? But otherwise to some extent you are just reinventing the wheel, if you are just going to set up something which deals hands with constraints and feeds it to a double-dummy solver.

 

The problem with relying on the bid/play/defense logic of your program, IMO, is that from what I have seen (from downloading your trial program), the logic can be pretty awful. Not at a level where I'm particularly interested in the results. Why would I care what works best against a bad computer player? I'd rather see someone do the effort to be able to feed generated hands and a specified auction into GIB's bridge.exe, and obtain single-dummy results, however slow that might be. This would especially be useful for game and slam hands, where DD analysis has flaws such as always counting suits such as AJx vs. KTx as zero losers, and picking up both onside Qs and offside doubleton queens on combos like AKJxx vs. xxx.

 

If you want to maintain whitespace formatting in pasted text, encase it in [ code] [ /code] tags without the space in the tag, compose the text with a monospace font.

 

By ease od input I do mean as simple as possible GUI. I certainly agree my program has play problems. What is the best single dummy play program available today. I will buy it and test it against my results. It may not be much different than my results but we have to test it to see. Thanks for the instruction on formatting spaces.

[antonylee]

By ease od input I do mean as simple as possible GUI. I certainly agree my program has play problems. What is the best single dummy play program available today. I will buy it and test it against my results. It may not be much different than my results but we have to test it to see. Thanks for the instruction on formatting spaces.

http://www.ny-bridge.com/allevy/computerbridge/index.htm

[Stephen Tu]

I wonder how GIB stacks up against current wbridge/Jack. Maybe I will manually run a challenge match between GIB & wbridge when I have some time.

[RossSCann]

I wonder how GIB stacks up against current wbridge/Jack. Maybe I will manually run a challenge match between GIB & wbridge when I have some time.

 

I don't expect my program to stand up well against professional products. I am an 80 year old programming hobbyist and low intermediate level bridge player. However, on large samples my program might yield similar results in total because of 2nd rate declarer play facing 2nd rate defense. Dealing with random deals on a single dummy basis is really hard programming. Nobody is ever going to write code capable of expert play because the effort that would be required can'r be justified on a cost/benefit basis. A DeepFineese approach using standard opening leads might work, however as a more accurate equivalent of expert play. My program has 280,000 lines of C++ code dealing single dummy play and that is just scratching the surface of what would be needed for advamced play/defense.

I wonder if any of the commercial programs have the facility to play an input file of N hands and print summary results?

If you do get the time to run your challenge match you should publish it here.

If anyone has a contact with one or more of the commercial programs you should ask them to make available a version that can run from an input file of N hands and print the results. The coding to do this would be trivial. There would be no reason to incorporate such a feature in their commercial product, but they could put the version on their web site if they are not afraid of the competition. They could even do it as a console app for simplicity, with no other capability than running the file of N hands and displaying results.

[broze]

I wonder how GIB stacks up against current wbridge/Jack. Maybe I will manually run a challenge match between GIB & wbridge when I have some time.

 

Does GIB even enter the computer-bridge world championships any more? And let me get this straight, Jackbridge is run and engineered by three amateur guys and GIB has a team of coders working for the most popular and lucrative bridge site in the world, yet is not even competitive. What am I missing?

[Stephen Tu]

Does GIB even enter the computer-bridge world championships any more? And let me get this straight, Jackbridge is run and engineered by three amateur guys and GIB has a team of coders working for the most popular and lucrative bridge site in the world, yet is not even competitive. What am I missing?

 

What you are missing:

- the brains behind GIB stopped development 10+ years ago in favor of his "day job" running a company he started. This is main reason why it hasn't entered world championship since. Also GIB's inventor had some beef with the contest coordinator which is why it didn't enter last 2/3 years when he was still working on it.

- BBO "team of coders" is devoted to mainly programming BBO site itself, not GIB. And integrating the robots with the site itself for the tournaments/robot races etc., not working on GIB's skill. AFAIK they have like 1 guy, maybe 2 working on GIB, only really know enough to work on bidding database tweaking, not to make advances in defense/play.

 

I will try version of GIB from 2 years ago (bid patches only, play engine still from 2002) vs. wbridge I downloaded earlier this year, maybe 64 board match, see if it is still competitive or not (I have not really played wbridge extensively to form opinion). I assume wbridge comparable to Jack since they have been close matches in WC and swapping places from year to year. Don't know how long this will take, it is tedious doing it manually.

[barmar]

Does GIB even enter the computer-bridge world championships any more? And let me get this straight, Jackbridge is run and engineered by three amateur guys and GIB has a team of coders working for the most popular and lucrative bridge site in the world, yet is not even competitive. What am I missing?

Team of coders? How big a company do you think BBO is? :) I'll bet one of those amateurs has more AI expertise than all of us put together (not difficult, since we have very little). We bought GIB from Ginsberg, and other than bidding rule improvements there have been very few changes to it. GIB is not our primary focus, BBO is.

[CarlRitner]

It would be interesting though, to see how GIB would fare with the best database version BBO has to date. I seem to recall the play engine wasn't all that bad. CPU speeds are way up, RAM is cheap. If BBO could legally enter GIB, it would be good news for bridge, very interesting to follow.

[broze]

Thanks Stephen, I see. Tbh I'm surprised BBO are content using software that, if it is not already outdated is doomed to become so with no one (or at least very few) working on the programming. Then again I suppose having the *best* robots is not really all that important for the functioning of the robot tournaments etc. I've been interested to read about the history of the site and how they've pioneered the online bridge scene. I just wonder what direction they're heading in next.

 

Best of luck with getting the match to work; I'll be interested to see the results.

[barmar]

We're not really content with it, it's just all we've got, and it's not so bad. It's not like we can just decide to replace GIB with some other program -- they have to be willing to license it to us.

[Stephen Tu]

I don't expect my program to stand up well against professional products. I am an 80 year old programming hobbyist and low intermediate level bridge player. However, on large samples my program might yield similar results in total because of 2nd rate declarer play facing 2nd rate defense. Dealing with random deals on a single dummy basis is really hard programming. Nobody is ever going to write code capable of expert play because the effort that would be required can'r be justified on a cost/benefit basis. A DeepFineese approach using standard opening leads might work, however as a more accurate equivalent of expert play. My program has 280,000 lines of C++ code dealing single dummy play and that is just scratching the surface of what would be needed for advamced play/defense.

I wonder if any of the commercial programs have the facility to play an input file of N hands and print summary results?

If you do get the time to run your challenge match you should publish it here.

If anyone has a contact with one or more of the commercial programs you should ask them to make available a version that can run from an input file of N hands and print the results. The coding to do this would be trivial. There would be no reason to incorporate such a feature in their commercial product, but they could put the version on their web site if they are not afraid of the competition. They could even do it as a console app for simplicity, with no other capability than running the file of N hands and displaying results.

 

It seems your program still has lots of issues. You still have a few hands that have 5-5 in the minors, which is impossible since North would rebid 2c, not 1nt. You have hands with North having 4 cd spades, which would have raised spades, not rebid 1nt. You have hands with 2245 and 1345 distribution that arguably would have opened 1c, not 1d, if the rebid planned was 1nt (if you open these hands 1d, it is with the intent to rebid 2c). You have hands with 16-17 hcp, which should not rebid 1nt (they should have opened 1nt, or rebid a new suit, possibly reversing). Your result matrix still has a handful of hands reaching 2nt/3nt contracts, which shouldn't be possible. You have something like 245 hands ending in 2H, which is too high given the distributional tally you posted. 2h should only be reached if North has 4 hearts, or 3 hearts and 1 spade. With equal length, or 2-3 in the majors, it's normal to give a preference. Following those rules with your deals, you'd only reach 2h 166 times. And that's before checking if your frequencies re HCP distribution and shape distribution are consistent with reality. After the 2h rebid, North should basically pass with 4 hearts or stiff spade, and bid 2s otherwise, which South would pass.

 

In any case, I don't think people other than yourself are going to really be interested in the output of your program. Why would we care how poor defense does vs. poor declarer play? I don't really have confidence that they would really cancel each other out, I'd rather just use double-dummy results. You say that "nobody is ever going to write code capable of expert play". I disagree with this. GIB *already* declares at an expert level IMO, at least after a few tricks when Gibson kicks in. Its problems are mainly in the bidding, and in the defense where its lack of signalling knowledge is a crippling handicap.

 

As for getting programs to get double-dummy/DeepFinesse results, there are already plenty of ways to do this. Thomas Andrew's Deal in Tcl, and Antony's port of this to python, already interface with Bo Haglund's double-dummy solver library, so you can write scripts to calculate results on a double-dummy basis. van Stavaren's dealer has code to interface with GIB's bridge.exe command line, which does have a double-dummy mode where it can read a hand from a file or stdin and output a result.

 

If you really wanted to program something that would be of use to people other than yourself, I would tackle the problem of feeding GIB a hand and a specified auction, and finding the *single dummy* result, then repeating for some large # of hands. This is theoretically possible using the console mode bridge.exe, but might be easier with access to the source code for the GUI. At one point there was a linux release of GIB including source code for the GUI, somebody may have a copy floating around somewhere. Or BBO could perhaps be convinced to re-release it. (I would also love it if they'd release mb.txt2 as Matt used to, the source of the bidding DB, would be useful for experiments). Maybe improving GIB's bidding DB could go open-source to some extent if BBO is willing, perhaps it would improve faster than what georgi can do on his own.