Wrong shape over preempt

[Cascade]

2008 World Mind Sports Final standard deviation was 5.88 IMPs per board.

 

GIB vs GIB playing the same hands the standard deviation was 6.17 IMPs per board.

 

Comments

 

1. GIB was slightly more variable than the players. So maybe it is true that world class players are less variable than GIB but I still think it is not obvious. GIB e.g. never misses a spot card, never gets tired, never goes on tilt etc etc all human failings that will contribute something to the variance.

 

2. When I have done similar calculations in the past my recollection is the standard deviation has been higher.

 

3. I actually forced GIB to play two different systems as I was worried that if it played the same system then that would reduce the variance (standard deviation). GIB played ACOL 4-card majors NS at one table and EW at the other table and Standard American 5-card majors in the other seats (other team).

 

4. For the record GIB Standard American beat GIB ACOL by 49 IMPs. Before you non Acolites get too excited GIB ACOL was actually leading before the start of the last set but GIB Standard American won 53 IMPs in the last set of 16 boards.

 

5. Given the difference between England and Italy was 30 IMPs (or 30/96 IMPs per board) and the standard deviation was 5.88 then if there was no difference between the two teams over 30% of the time Italy would have won by a larger margin. Basically this means the result is not statistically significant. In order for the result to be significant at a 5% level if the true difference in the teams was 5/16 IMP per board then around 1000 deals would need to be played. At 1% significance we would need something over 1900 deals.

 

As an aside someone once told me that when Terrance Reese was asked how many deals would be needed to conclusively decide which team was better he replied 10000 (or something similar). Does anyone have a reference for a quote of this sort?

 

6. The arguments about the variance aren't really important in my view. If the variance is higher then we just need to do a bigger simulation so that the randomness from the variance is overcome and we can make some conclusions about the skill factor.

 

7. Small simulations as I have reported in this thread seldom prove anything - although if you get differences of the order of 1.5 IMPs per board as I reported the results are pretty conclusive after a few hundred deals.

 

8. The question we would like to prove is "Is overcalling 2♠ superior to passing?". All we actually prove with simulations like this is that GIB bidding 2♠ did better than GIB passing. It is possible there is something other than what is inate in the problem in the GIB engine that biases the results one way or another so that the extrapolation from GIB to real world is completely invalid. However by and large despite the many anecdotes that suggest otherwise GIB plays a reasonable game and its judgement actions are based on simulations which even when not good will simply increase the variance so we will have to make a bigger simulation. Basically the GIB simulation produces odd actions when it gets a bad sample - this is just like a human player making a mistake. It is just that GIBs mistakes are different than human mistakes.

 

9. If you could get to a world championship final repeatable actions that are 1.5 IMPs per board better than your opponents will see you win comprehensively.

 

10. Probably some other things but I am too tired to think about them ... maybe more later ...

 

11. I actually did another 100 hands with replicated the nature of the results I reported above but in this third set of 100 hands I also considered overcalling 2NT and overcalling 3♣. In this set of 100 hands the order from best to worst actions was:

 

2♠ +560

3♣ +332

Dbl -95

2NT -256

Pass -541

 

The standard deviation in for example the head to head 2♠ versus Pass was almost 8 IMPs per board which is higher than before and is to be expected when you force a variation between the two auctions. That would mean that a larger sample was needed to draw reliable conclusions. Head to head 2♠ was up over 2 IMPs per board on the 100 boards in this last sample.

[Fluffy]

I loved reading this stuff Wayne.

 

Go canape!, always the major over the longr minor :P.

[shevek]

I would rather kill myself than bid 2S.

No need to get personal.

 

I pass too but you could extend ELC doubles to auctions like that. One ELC princiiple is that they only apply when partner bids a minor, such as

1♠ - X - no - 2♣ - no - 2♦ = 4-5 reds. Or

1♣ - X - no - 1♦ - no - 1♥ = 4-5 majors.

 

You could try extending to auctions like this, when

2♦ - X - no - 2♥ - no - 2♠ = 4-5 blacks.

 

Not 5-4 the other way because that hand overcalls. I quite like that but partner's never agree to it, pointing out that partner might bid 3♥ or 4♥.

 

These sort of flexible doubles are more use to strong clubbers.

Change it to a 19-count and a different auction

♠AKxx  ♥Ax  ♦Kx  ♣AJxxxx

 

1♣* - 3♦ - no - no - ?

 

I like double then 3♠ to show this.

[se12sam]

Wayne, The analysis is brilliant. Thank you very much.

[Cascade]

...

 

12. GIB is probably biased against four-card overcalls as I imagine it is programmed to assume a five-card overcall when doing its simulations from partner's perspective. Obviously it sees its own cards when doing the simulations from the overcalling side and judges accordingly.

[awm]

My observations have been that:

 

(1) GIB has trouble penalty-doubling partscores. It does not make these doubles very often, and when it does the reason for doing so often seems sort of random. It also removes penalty doubles of partials fairly often for no obvious reason. GIB also makes a lot of defensive errors against partscore contracts (more than it makes against games or slams). Since one of the big downsides to acting on this hand is the possibility of a penalty double from LHO, this tends to help bidding/doubling disproportionately over passing.

 

(2) GIB responds to takeout doubles in very strange ways. I have seen it bid three-card suits in preference to four-card suits (even when 4-3 in the majors over a takeout double of a minor). It will do things like insist on a four-card suit after partner's double for no reason, or convert a takeout double to penalty with no real hope of setting. This tends to make doubling look worse than it would opposite a player who knows how to respond to takeout doubles.

 

(3) GIB seems to have determined (rightly or wrongly) that we need to get into the auction on this hand. Subject to that determination, it is probably safer to bid right away than to pass and then balance into a four-card spade suit over 3♦ or 4♦. Thus we are not really comparing "bid 2♠ now" against "pass throughout" -- we are comparing "bid 2♠ now" to "pass now and balance into spades at the three or four level." It's not particularly surprising that bidding 2♠ now wins. Obviously I could be wrong about this, but you did mention several hands where GIB passed and then backed in with some ridiculous call later on.

 

(4) Obviously the results depend heavily on which hands are eligible for a 2♦ preempt and how aggressively partner will balance or pre-balance in auctions like 2♦-P-P and 2♦-P-3♦.

[mike777]

Interesting thread I hope to hear more.

[Cascade]

(1) GIB has trouble penalty-doubling partscores. It does not make these doubles very often, and when it does the reason for doing so often seems sort of random. It also removes penalty doubles of partials fairly often for no obvious reason. GIB also makes a lot of defensive errors against partscore contracts (more than it makes against games or slams). Since one of the big downsides to acting on this hand is the possibility of a penalty double from LHO, this tends to help bidding/doubling disproportionately over passing.

Here are all of the hands from one set of 100 where GIB hand 10+ HCP and four or more spades sitting over the 2♠ bidder and an analysis of what happened.

 

Draw your own conclusions.

 

Some observations:

 

1. I wouldn't have doubled with all of these hands. Especially at IMPs I am a bit of a wimp with the axe.

 

2. While the big penalties +500, +800 etc that were missed look bad there will be more -670s etc when you consider if you double with all of these hands you will also be doubling with similar hands when the overcaller has a five or six-card suit and therefore probably will be able to take more tricks.

 

3. GIB only doubled five times. The weakest with an 11-count. There are 19 times when it had similar hands and did not double. Not all of those doubles would have been successful.

 

Here are the hands that GIB doubled on after 2♦ (2♠) ...

 

JT98.KQT98.98.AQ -670 when 2♠ made.

 

QT84.KJT7.54.AKJ +800

 

QJT7.KJ73.Q5.KQ7 +200

 

QJT64.KT85.A6.K7 +1100 (There were at least 8 tricks available in hearts)

 

QJ954.KT7.A5.QT6 +500

 

Here are the hands were GIB did not double when it might have ...

 

QJ965.KJT.T7.AQJ GIB bid 3♦ +500 was available in 2♠X

 

QJ95.KQJ3.AQ.QT6 GIB bid 3♦ - insane not only did they not get the +1400 or so penalty but they didn't get to their cold game.

 

J964.974.QT96.AK GIB bid 4♦ and beat 4♠ by one trick.

 

J9864.KJ.T73.AKJ GIB bid 3♦ and doubled 3♠ for +500

 

T985.QJ43.AJ8.A6 GIB bid 3♦ and 3♠ made

 

QJ985.J64.Q93.KQ GIB bid 3♦ - 2♠ is probably 1-off (its not 2-off)

 

J9875.J86.4.AKJ6 GIB Passed - 2♠ went 2-off so 500 was available

 

Q865.Q93.765.AKT GIB bid 3♦ - 2♠ is no more than one-off

 

JT86.KQJ95.87.K6 GIB Passed 2♠ went 1-off

 

QT64.K984.73.KQ6 GIB Passed 2♠ made

 

J8654.KQ98.J4.K6 GIB Passed 2♠ made an overtrick

 

QT74.43.A64.AQ75 GIB raised to 3♦ NS can play 3♥ they got to 4♥ down one.

 

Q864.K7543.6.AK6 GIB passed 2♠ went 3-off

 

QT9865.K6.Q.AKQ6 GIB passed which became the final contract. NS can make game in hearts which they did after an initial double. 2♠ was also the final contract after a Pass with the hand in the opening post as GIB apparently can bid 2♠ non-forcing which it did in response to the weak two.

 

QT876.J7.T87.AKQ GIB bid 3♦ which made. NS can play hearts in a 7-2 fit.

 

J976.K9643.Q.AJ6 GIB passed 2♠ went 1-off.

 

QJ864.QJ3.AT5.T5 GIB raised to 3♦. NS can play hearts which they did - 4♥ 1-off (maybe 3♥ making after a penalty double.)

 

9765.KQJ953.A.A6 GIB passed. 2♠ went four-off

 

T876.KQT94.A3.Q5 GIB bid 3♦ which made. 2♠ probably down one maybe down two.

[awm]

Well, suppose a simple rule like "double if you have two spade tricks assuming RHO has all the remaining high ones, and additionally you have 12+ overall hcp."

 

Under such a rule:

 

GIB would not double on the hand where 2♠X made, where it had only one trump trick.

 

GIB would still double on all the hands where 2♠X failed.

 

GIB would double on five of the hands where it did not double. On all of these, 2♠X would go off (in some cases several tricks). On two of these hands GIB obtains a much better result than it did at the table. On one hand GIB actually got to double 3♠ by passing for a bigger number, and on the other two there is some superior resting place for the other side which may or may not be reached after the penalty double of 2♠.

 

So it is the case that GIB made at least one bad double and missed at least a few opportunities for good doubles.

[Cascade]

A quick look suggests that a 12 count with four trumps (including some trump tricks) will allow 2♠ to make somewhere between 35% and 40% opposite a standard 6-9 hcp range weak two.

 

The opening post suggested this pair were "frisky". Adjusting the range to 5-8 hcp increased the makeable 2♠ to nearly 60%.

 

In both cases sometimes with overtricks.

 

All of this in the case of a more normal 2♠ overcall with 5-6 spades and 11-16 HCP.

 

I mean it is not surprising that a frisky double works out when the overcaller turns up with only a 4-bagger.

 

The real boundary for light-ish doubles will depend on how they preform on the more normal and more frequent five-card or longer overcalls.

[awm]

Regardless of your criteria, are you looking at some of these hands:

 

♠JT98 ♥KQT98 ♦98 ♣AQ

 

This is a crazy double. You have four tricks at best (two clubs, heart, spade). I don't think any human player would make a penalty double on this hand over 2♦-2♠. And sure enough, GIB's double was a huge IMP loss when 2♠X made.

 

♠QJ965 ♥KJT ♦T7 ♣AQJ

 

This is a pretty obvious double. You have three potential trump tricks, and additional sources of defense in hearts and clubs, and only two diamonds. Perhaps if partner typically preempts on utter trash you might consider a pass, but 3♦?

 

♠QJ95 ♥KQJ3 ♦AQ ♣QT6

 

You even admit GIB's failure to double (or at least bid game) here was crazy. Two trump tricks and seventeen points not enough to double? Where are declarer's tricks? Maybe four trumps and the heart ace and the club ace king (giving partner KJxxxx of diams and out) and 2♠ is still one off.

 

These are the three that seem really obvious to me. I don't think any good player would make the decisions GIB made, and these three decisions lost on the order of 30 imps.

 

Surely GIB makes mistakes in other auctions as well, but I don't think the frequency of mistakes as responder after 2♦-Pass is going to be all that high because GIB may not even be "out of book" yet on that auction. And it sounds like GIB was not infrequently making crazy calls with the given hand after 2♦-Pass-3♦-Pass-Pass either.