Tests for a double-dummy solver

[awm]

Here's a fairly simple test I'd like to see on a double dummy solver.

 

Suppose south has a particular distribution. We can generate thousands of random hands under this constraint. What is the probability that N/S can make a game?

 

This sort of test will give us some idea of the value of shape. One of the main points ZAR makes is that 6-3-3-1 shape is more powerful than 6-3-2-2 (14 vs. 13 ZAR points, ignoring high cards). This test will tell us whether this is true, and how large the effect is.

 

I would expect some interesting results. For example, it wouldn't surprise me if 6-3-3-1 (in that order) was better than 6-3-2-2, but 3-3-1-6 was not substantially better than 3-2-2-6. The reasoning being that the singleton is useful when you play in a suit, but the hands with a long minor are more likely to play in notrump where a singleton or doubleton makes little difference. In addition, I wouldn't be surprised to see that 4-4-3-2 is substantially better than 4-2-3-4, due to the increased chances of a major suit fit.

[han]

It seems to me that Tysens work on binky points almost answers your questions, although I don't think that they distinguish between majors and minors. An interesting addition.

[pdmunro]

Richard Pavlicek has a huge database of hands that he has solved double dummy. Go to http://www.rpbridge.net/rpbr.htm and look under "Bridge Utilities" for "RP Deal Finder". The database can be interrogated with questions such as yours.

[tysen2k]

I don't have the % of time it makes game, but I do have the average number of tricks that each pattern makes. It doesn't differentiate between majors and minors.

 

4-3-3-3  7.80
4-4-3-2  8.09
5-3-3-2  8.14
5-4-2-2  8.41
6-3-2-2  8.51
4-4-4-1  8.62
5-4-3-1  8.69
6-3-3-1  8.78
7-2-2-2  8.91
6-4-2-1  9.02
5-5-2-1  9.03
7-3-2-1  9.14
5-4-4-0  9.38
5-5-3-0  9.51
6-4-3-0  9.51
8-2-2-1  9.57
6-5-1-1  9.61
7-3-3-0  9.65
7-4-1-1  9.67
8-3-1-1  9.83
6-5-2-0  9.88
7-4-2-0  9.89

 

But the suggestion you make is an interesting one. I think I could whip it up and take a look at the %game statistic. That would differentiate between majors and minors and see how much a 1444 hand is compared to a 4441 hand. Good idea.

[tysen2k]

Okay, here we go:

 

Shape    Game?
2-2=5-4  24%
3-2=4-4  24%
3-3=4-3  25%
4-3=3-3  25%
3-2=5-3  26%
4-2=4-3  26%
3-3=5-2  27%
4-3=4-2  27%
2-2=6-3  27%
4-4=3-2  28%
3-2=6-2  28%
4-2=5-2  28%
5-2=3-3  28%
5-3=3-2  28%
5-2=4-2  29%
3-1=5-4  29%
2-1=6-4  30%
2-1=5-5  30%
4-1=4-4  31%
3-1=6-3  31%
5-4=2-2  31%
4-1=5-3  32%
3-3=6-1  32%
6-2=3-2  32%
6-3=2-2  33%
4-3=5-1  33%
5-1=4-3  34%
4-4=4-1  34%
5-3=4-1  35%
4-2=6-1  35%
4-1=6-2  35%
5-4=3-1  37%
6-1=3-3  37%
6-3=3-1  38%
5-2=5-1  38%
5-1=5-2  39%
6-1=4-2  40%
6-2=4-1  41%
4-0=5-4  42%
6-4=2-1  43%
5-5=2-1  43%
5-0=4-4  44%
5-4=4-0  49%

 

Edit: A more complete list is now available on the 2nd page of this thread.

 

Because of what we're looking for, S&H are equivelent, as are D&C. So I'm going to use the notation A-B=C-D to be A&B in the majors and C&D in the minors (either way for both).

 

So maybe 2245 is the worst distribution in bridge and not 3334! ;)

 

Okay, what can we make of this? It's a very loose scale, but 1 point of "normal" distribution corresponds to about 3.8% game increase. On Zar's scale, it's about 1 point per 2.3%.

 

So that means that the difference between 4-4=4-1 and 4-1=4-4 is about 1.0 points or 1.6 Zar.

 

To address awm's original guesses, the difference between 6-3=3-1 and 6-3=2-2 is about the same as the difference between 3-3=6-1 and 3-2=6-2 (about 1.2 points or 2.0 Zar in both cases). And the difference between 4-4=3-2 and 4-2=4-3 is only 0.5 points or 0.8 Zar.

 

There are some other interesting ramifications here, but I don't want to hog the stage. Anyone want to comment?

 

Tysen

[han]

Yes I do want to comment:

 

COOL!!!!

[Guest Jlall]

6-4-2-1 and 5-5-2-1 are less likely to make a game than 5-4-4-0? I find that really shocking. This is some interesting stuff.

[han]

Is it hard to add some of the freaky distributions?

 

I'm missing 5-0-4-4, that should do worse than 5-4-4-0.

[han]

6-4-2-1 and 5-5-2-1 are less likely to make a game than 5-4-4-0? I find that really shocking. This is some interesting stuff.

Shocking indeed, and notice the following:

 

6-4-2-1: 15 Zar points, and 4 5-3-1 points.

 

5-5-2-1: 14 Zar points, and 4 5-3-1 points.

 

5-4-4-0: 14 Zar points, and 5 5-3-1 points.

[Blofeld]

Mmm, I find it interesting that the only distribution shown with a void scores a way above the rest - I too would be interested in seeing some more freak distributions.

 

Many thanks for this, Tysen - it's amazing to be able to look through that table.

[han]

Does this table present a good case for opening light in the majors but more soundly in the minors?

 

I think so, at least for shapely hands. The difference between 6-3-3-1 and 1-3-3-6 is 6%, the same as the difference between 6-3-3-2 and 2-3-3-6. Does this mean that we should have almost 2 HCP or 3 Zar points more to open with these shapes when the long suit is a minor?

 

I guess this argument is a bit too simplistic, but it sure is interesting.

[Echognome]

For starters, I wanted to say that I think this work might be quite useful for any hand evaluation methods and applaud the efforts.

 

A question for Tysen though is how are these probabilities calculated?

 

Is it Prob(10+ tricks in a major U 11+ tricks in a minor U 9+ tricks in NT | My hand shape = X) = Prob(10+ tricks in a major U 11+ tricks in a minor U 9+ tricks in NT| my hand shape = X AND my hcp = 0)Prob(my hcp = 0| my hand shape = x) + Prob(.... | my hand shape = X AND my hcp = 1)Prob(my hcp = 1 | my hand shape = X) + ... etc.

 

For this conditioning argument are the conditional high card point frequencies used? That is to say, we know that shape and hcp are correlated (since with freak distributions there are less empty spaces to hold high cards).

 

Perhaps this is all intrinsic when you simulate enough hands?

[Guest Jlall]

sorry I've never been good at counting points in traditional ways, but I thought 5-3-1 points only applied once we had a 9 card fit? Like... I would prefer 5-4-4-0 to 5-5-2-1 if partner opened 1H so it makes sense. But I would expect 5-5-2-1 to have a better chance a priori to make a game than 5-4-4-0.

[awm]

Thanks Tysen! Great job.

 

This seems to support the idea that it's good to open more aggressively with length in the majors. Neither ZAR nor binky nor any scheme I've seen really deals with this well -- for example 2254 and 5422 would evaluate the same under most schemes but yet the chance of game on the second hand is substantially greater. It seems very reasonable to open lighter with 5422 shape than with 2254, without even considering the obstructive value of a 1♠ opening bid.

 

Another point here is the power of voids. Methods like the "rule of 20" for opening tend to consider 5521 shape as "stronger" than 5440. ZAR distribution points (before fit is known) also agree, giving 15 points for the first and 14 for the second. But in fact it seems that 5440 shape is more likely to produce a game. This suggests that the 5/3/1 scheme has something going for it, but it's hard to see from the percentages how points and shape interact.

 

Perhaps a reasonable experiment would be to say, given a particular distribution, how many bumrap points (or points+controls as they are roughly the same) do you need such that your probability of making game is at least P (say P=40% as a reasonable threshold to start with)? This might give us a grasp on the "value" of these different shapes, which could eventually lead to something better than a 5/3/1 type approach.

[mike777]

Thanks Tysen! Great job.

 

This seems to support the idea that it's good to open more aggressively with length in the majors. Neither ZAR nor binky nor any scheme I've seen really deals with this well -- for example 2254 and 5422 would evaluate the same under most schemes but yet the chance of game on the second hand is substantially greater. It seems very reasonable to open lighter with 5422 shape than with 2254, without even considering the obstructive value of a 1♠ opening bid.

 

Another point here is the power of voids. Methods like the "rule of 20" for opening tend to consider 5521 shape as "stronger" than 5440. ZAR distribution points (before fit is known) also agree, giving 15 points for the first and 14 for the second. But in fact it seems that 5440 shape is more likely to produce a game. This suggests that the 5/3/1 scheme has something going for it, but it's hard to see from the percentages how points and shape interact.

 

Perhaps a reasonable experiment would be to say, given a particular distribution, how many bumrap points (or points+controls as they are roughly the same) do you need such that your probability of making game is at least P (say P=40% as a reasonable threshold to start with)? This might give us a grasp on the "value" of these different shapes, which could eventually lead to something better than a 5/3/1 type approach.

Ok, how do you conclude this? Why is it better to open aggressively with length in majors compared to minors? I assume there are just as many advantages to opening agrressively in the minors, how does this refute that in favor of the majors? We all know it takes one less trick in the majors but so what? The chance for game in 8 or 9 card major is almost always greater than the same in a minor with same number of cards, so what?

 

I see the data but I do not see this conclusion, can anyone help?

[Blofeld]

The two advantages to opening aggressively in the majors more than in the minors (that I can see) are:

1) 1M is more preemptive than 1m. We knew this already.

2) Hands with major length are more likely to have a game on with the same HCP strength. We also knew this already (as a matter of common sense), but these data help to quantify the effect. So passing on the borderline minor hands has less risk of missing a game than on the same hands with majors instead.

[tysen2k]

Is it hard to add some of the freaky distributions?

 

I'm missing 5-0-4-4, that should do worse than 5-4-4-0.

5-0=4-4 was cut off from the list. It is 44%. I'll edit the original post to include it.

 

I cut off from the table everything that happened less than 5000 times in my database. So 5-0=4-4 just happened to be below that while the other 5440 patterns were above. At a count of 5000, the error is on the order of 0.5%, so I didn't want to include rarer patterns. Let me know if any other patterns are missing.

 

Tysen

[tysen2k]

Is it Prob(10+ tricks in a major U 11+ tricks in a minor U 9+ tricks in NT | My hand shape = X) = Prob(10+ tricks in a major U 11+ tricks in a minor U 9+ tricks in NT| my hand shape = X AND my hcp = 0)Prob(my hcp = 0| my hand shape = x) + Prob(.... | my hand shape = X AND my hcp = 1)Prob(my hcp = 1 | my hand shape = X) + ... etc.

Yes. And while HCP & Shape are slightly correlated, when you add up every single HCP possibility, you get everything.

[mike777]

The two advantages to opening aggressively in the majors more than in the minors (that I can see) are:

1) 1M is more preemptive than 1m. We knew this already.

2) Hands with major length are more likely to have a game on with the same HCP strength. We also knew this already (as a matter of common sense), but these data help to quantify the effect. So passing on the borderline minor hands has less risk of missing a game than on the same hands with majors instead.

But is this not a function of majors being higher ranked and needing only 10 tricks. Again I see no evidence that an aggressive one level minor suit cannot be a very effective preempt effect at the one level. Virtually as much if not the same as a major. I see no reason to not open one level minors aggressively based on any of this data, can someone clue me in? thanks in advance.

[awm]

Ok, how do you conclude this? Why is it better to open aggressively with length in majors compared to minors? I assume there are just as many advantages to opening agrressively in the minors, how does this refute that in favor of the majors? We all know it takes one less trick in the majors but so what? The chance for game in 8 or 9 card major is almost always greater than the same in a minor with same number of cards, so what?

 

I see the data but I do not see this conclusion, can anyone help?

There are some underlying assumptions here I may not have mentioned. Perhaps the main question is: why should I open the bidding, as opposed to passing? Some reasons:

 

(1) If I think we might have a game, I should bid so we can reach that game.

 

(2) In order to get in the opponents' way, to make it harder for them to find a contract.

 

(3) If I think we can make a partscore, perhaps I should open so we can get there.

 

(4) In order to help partner on defense, to find the right lead, count my pattern, etc.

 

All of these are perfectly fair reasons for bidding. But assuming fairly "constructive" methods, it seems like (1) is the major reason for opening at the one level. Keep in mind that one-level bids don't steal a huge amount of space from the opponents. Of course, things change a little bit in 3rd seat (where 3 and 4 become bigger concerns) and in 4th.

 

Note that many systems seem to base an opening bid on "I have half what we need for game." We see this with Goren (26 hcp for game, 13 to open), with LTC (14 losers for major suit game, 7 losers to open), and with ZAR (52 for game, 26 to open). All of these seem to be working on the assumption that (1) is the major reason to open.

 

So if we're willing to assume that the main reason for opening is to find our games, it seems like an opening should announce that game is reasonably likely given opener's hand. This is really what the methods above are going for isn't it? So that seems to support opening lighter with major suit length.

 

To give a simple example, suppose I am deciding whether to open a balanced eleven count in first seat. Since I'm balanced, directing a lead from partner (condition 4) isn't a big deal. Since I play fairly standard methods, my opening on any (4432) pattern will be one of a minor, which doesn't really take any space from the opponents. So the only real concern here is, what do I think are our chances at game? It seems from Tysen's data that it might be reasonable for me to open a 4-4-2-3 eleven count, but that with a 2-3-4-4 eleven I should probably pass.

[Al_U_Card]

The two advantages to opening aggressively in the majors more than in the minors (that I can see) are:

1) 1M is more preemptive than 1m. We knew this already.

2) Hands with major length are more likely to have a game on with the same HCP strength. We also knew this already (as a matter of common sense), but these data help to quantify the effect. So passing on the borderline minor hands has less risk of missing a game than on the same hands with majors instead.

But is this not a function of majors being higher ranked and needing only 10 tricks. Again I see no evidence that an aggressive one level minor suit cannot be a very effective preempt affect at the one level. Virtually as much if not the same as a major. I see no reason to not open one level minors aggressively based on any of this data, can someone clue me in? thanks in advance.

Were they not referring to game producing potential? Unless the suit rank is considered and the higher ranking suit(s) get more bang for their buck?

[EricK]

Is this a reasonable question:

 

Given each particular hand shape, how many points are required so that the chance of game is above some particular threshold? (But what threshold?)

 

Will this give you some idea of how much more you need to open a minor suit hand than a major suit hand?

 

Eric

[mike777]

Ok, how do you conclude this? Why is it better to open aggressively with length in majors compared to minors? I assume there are just as many advantages to opening agrressively in the minors, how does this refute that in favor of the majors? We all know it takes one less trick in the majors but so what? The chance for game in 8 or 9 card major is almost always greater than the same in a minor with same number of cards, so what?

 

I see the data but I do not see this conclusion, can anyone help?

There are some underlying assumptions here I may not have mentioned. Perhaps the main question is: why should I open the bidding, as opposed to passing? Some reasons:

 

(1) If I think we might have a game, I should bid so we can reach that game.

 

(2) In order to get in the opponents' way, to make it harder for them to find a contract.

 

(3) If I think we can make a partscore, perhaps I should open so we can get there.

 

(4) In order to help partner on defense, to find the right lead, count my pattern, etc.

 

All of these are perfectly fair reasons for bidding. But assuming fairly "constructive" methods, it seems like (1) is the major reason for opening at the one level. Keep in mind that one-level bids don't steal a huge amount of space from the opponents. Of course, things change a little bit in 3rd seat (where 3 and 4 become bigger concerns) and in 4th.

 

Note that many systems seem to base an opening bid on "I have half what we need for game." We see this with Goren (26 hcp for game, 13 to open), with LTC (14 losers for major suit game, 7 losers to open), and with ZAR (52 for game, 26 to open). All of these seem to be working on the assumption that (1) is the major reason to open.

 

So if we're willing to assume that the main reason for opening is to find our games, it seems like an opening should announce that game is reasonably likely given opener's hand. This is really what the methods above are going for isn't it? So that seems to support opening lighter with major suit length.

 

To give a simple example, suppose I am deciding whether to open a balanced eleven count in first seat. Since I'm balanced, directing a lead from partner (condition 4) isn't a big deal. Since I play fairly standard methods, my opening on any (4432) pattern will be one of a minor, which doesn't really take any space from the opponents. So the only real concern here is, what do I think are our chances at game? It seems from Tysen's data that it might be reasonable for me to open a 4-4-2-3 eleven count, but that with a 2-3-4-4 eleven I should probably pass.

Excellent post, very well written.

 

You have shifted the debate to the more important question. Why do we open?

You have listed 4 excellent reasons and stated there may be other good reasons.

 

Here are a couple more reasons off the top of my head.

5) opening lite makes balancing decisions easier, why, because we do not have to balance, we have told our story.

 

6) Not only are we making it harder for the opp to bid their contracts (2) but it can be quite exhausting to have active opp bidding. This takes time and energy away from their declarer and defensive play.

 

For sake of discussion let's agree reason number one, constructive bidding, is the most important.

 

Of course agreeing it is the most important does not mean it is of overriding importance. How much constructive bidding are we losing compared to the gain for the other reasons. By adding up several other reasons their combined importance may conclude we should open minors or 11 hcp balanced hands aggressively.

 

Bottom line I do not see the data answering this difficult question.

[mike777]

The two advantages to opening aggressively in the majors more than in the minors (that I can see) are:

1) 1M is more preemptive than 1m. We knew this already.

2) Hands with major length are more likely to have a game on with the same HCP strength. We also knew this already (as a matter of common sense), but these data help to quantify the effect. So passing on the borderline minor hands has less risk of missing a game than on the same hands with majors instead.

But is this not a function of majors being higher ranked and needing only 10 tricks. Again I see no evidence that an aggressive one level minor suit cannot be a very effective preempt affect at the one level. Virtually as much if not the same as a major. I see no reason to not open one level minors aggressively based on any of this data, can someone clue me in? thanks in advance.

Were they not referring to game producing potential? Unless the suit rank is considered and the higher ranking suit(s) get more bang for their buck?

I repeat this is a function of suits needing only 10 tricks for game, so what? What is the new point?

[tysen2k]

Okay here is a more complete list:

 

Shape   Game?    Error
2-2=5-4  24%     0.3%
3-2=4-4  24%     0.2%
3-3=4-3  25%     0.2%
4-3=3-3  25%     0.2%
3-2=5-3  26%     0.2%
4-2=4-3  26%     0.1%
3-3=5-2  27%     0.2%
4-3=4-2  27%     0.1%
2-2=6-3  27%     0.4%
4-4=3-2  28%     0.2%
3-2=6-2  28%     0.3%
4-2=5-2  28%     0.2%
5-2=3-3  28%     0.2%
5-3=3-2  28%     0.2%
5-2=4-2  29%     0.2%
3-1=5-4  29%     0.3%
2-2=7-2  29%  *  0.7%
2-1=6-4  30%     0.4%
2-1=5-5  30%  *  0.5%
4-1=4-4  31%     0.3%
3-1=6-3  31%     0.4%
5-4=2-2  31%     0.3%
2-1=7-3  32%  *  0.7%
4-1=5-3  32%     0.3%
3-3=6-1  32%  *  0.5%
6-2=3-2  32%     0.3%
6-3=2-2  33%     0.4%
4-3=5-1  33%     0.3%
5-1=4-3  34%     0.3%
2-2=8-1  34%  *  2.2%
3-1=7-2  34%  *  0.7%
4-4=4-1  34%     0.3%
5-3=4-1  35%     0.3%
3-2=7-1  35%  *  0.7%
4-2=6-1  35%     0.5%
4-1=6-2  35%     0.4%
5-4=3-1  37%     0.3%
2-1=8-2  37%  *  1.6%
6-1=3-3  37%  *  0.5%
6-3=3-1  38%     0.4%
5-2=5-1  38%     0.4%
7-2=2-2  38%  *  0.8%
1-1=6-5  38%  *  1.2%
5-1=5-2  39%     0.4%
3-0=5-5  39%  *  1.1%
6-1=4-2  40%     0.5%
1-1=7-4  40%  *  1.7%
6-2=4-1  41%     0.5%
3-0=6-4  41%  *  0.9%
4-0=5-4  42%  *  0.6%
6-4=2-1  43%     0.5%
4-0=6-3  43%  *  0.9%
3-0=7-3  43%  *  1.4%
7-2=3-1  43%  *  0.7%
7-1=3-2  43%  *  0.7%
5-5=2-1  43%  *  0.6%
7-3=2-1  44%  *  0.8%
5-0=4-4  44%  *  0.9%
4-1=7-1  44%  *  1.2%
3-3=7-0  44%  *  2.0%
4-4=5-0  45%  *  0.9%
3-1=8-1  45%  *  2.1%
2-0=7-4  45%  *  1.7%
2-0=6-5  45%  *  1.3%
4-3=6-0  45%  *  0.9%
5-0=5-3  46%  *  0.8%
1-1=8-3  46%  *  2.9%
3-0=8-2  47%  *  3.1%
5-3=5-0  48%  *  0.8%
5-1=6-1  48%  *  0.9%
5-4=4-0  49%  *  0.7%
2-0=8-3  50%  *  3.1%
6-0=4-3  50%  *  0.9%
6-3=4-0  51%  *  0.9%
4-2=7-0  51%  *  1.7%
3-2=8-0  51%  *  3.2%
6-1=5-1  51%  *  0.9%
8-1=2-2  52%  *  2.2%
4-0=7-2  52%  *  1.7%
5-5=3-0  52%  *  1.1%
6-4=3-0  52%  *  0.9%
8-2=2-1  53%  *  1.7%
5-2=6-0  53%  *  1.3%
7-0=3-3  54%  *  1.9%
5-0=6-2  55%  *  1.3%
1-0=7-5  55%  *  3.0%
6-2=5-0  56%  *  1.3%
7-3=3-0  57%  *  1.4%
6-0=5-2  57%  *  1.3%
7-1=4-1  57%  *  1.2%
4-1=8-0  57%  *  4.6%
1-0=6-6  58%  *  3.7%
6-5=1-1  58%  *  1.2%
7-4=1-1  59%  *  1.6%
4-0=8-1  59%  *  4.7%
7-0=4-2  60%  *  1.7%
7-2=4-0  60%  *  1.7%
8-0=3-2  60%  *  3.1%
8-1=3-1  61%  *  2.1%
6-5=2-0  61%  *  1.2%
1-0=8-4  63%  *  4.7%
7-4=2-0  63%  *  1.6%
8-2=3-0  63%  *  3.0%
5-0=7-1  64%  *  2.8%
8-3=1-1  65%  *  2.7%
5-1=7-0  65%  *  3.0%
8-3=2-0  67%  *  2.9%
6-1=6-0  70%  *  2.5%
7-0=5-1  70%  *  2.8%
6-0=6-1  71%  *  2.4%
7-5=1-0  72%  *  2.9%
8-0=4-1  73%  *  4.3%
7-1=5-0  73%  *  2.8%
8-1=4-0  74%  *  4.0%
8-4=1-0  75%  *  4.3%
6-6=1-0  83%  *  3.1%

 

If there is a "*" that means the error is more than 0.5% and so the value should be looked at with a grain of salt.

 

Look at some of the ones that were excluded before. 2-2=7-2 stands out like a sore thumb. It rarely produces a game and is worse than all 4441 shapes.