Team match chatGPT versus Bart

[helene_t]

[hv=https://www.bridgebase.com/tools/handviewer.html?lin=st||pn|thepossum,~Mwest,~Mnorth,~Meast|md|1ST8HTDKJ9863CAQJ8,S62HA865432DT75CK,SQJ53HQ97DAQ2CT94,SAK974HKJD4C76532|sv|e|rh||ah|Board%203|mb|1D|an|Minor%20suit%20opening%20--%203+%20!D;%2011-21%20HCP;%2012-22%20total%20points|mb|2H|an|Aggressive%20weak%20jump%20overcall%20--%206+%20!H;%204-10%20HCP|mb|D|an|Negative%20double%20--%204+%20!S;%207+%20HCP;%208+%20total%20points|mb|2S|an|14+%20total%20points;%20forcing%20to%203H|mb|3C|an|4+%20!C;%204+%20!D;%203-%20!S;%2021-%20HCP;%2015-22%20total%20points|mb|P|mb|3N|an|4-%20!H;%204-5%20!S;%2012-19%20HCP;%20stop%20in%20!H|mb|P|mb|P|mb|P|pc|HK|pc|HT|pc|H6|pc|H7|pc|HJ|pc|S8|pc|H8|pc|HQ|pc|DA|pc|D4|pc|D3|pc|D5|pc|CT|pc|C2|pc|CA|pc|CK|mc|11|]400|300[/hv]

 

I was currious if our future overlords could bid those hands better than we mortals, so I let Bard sit south and chatGPT west. I took the North and East seat myself. The auctions proceeded:

pass-1♦-pass-1♠

2♦-3♦-dbl-a.p.

 

Note that it was W who opened 1♦ - the auction would have made a bit more sense if we rotated it 90 degress so that it was South that opened 1♦.

 

At the other table, same setup but now chatGPT South and Bard West, it went

 

1♥-2♥-2♠-4♠

dbl-a.p.

 

Again, note that it was South who opened 1♥.

 

Their reasoning for their bids was absolutely nonsense, contradicting themselves, thinking that 12 points is more than 14 points etc. chatGPT, in a single paragraph, first recited his hand correctly but then corrected himself by noting that he had been dealt 20 cards, which he apparently found normal as his version of SAYC apparently has systematic bids for hands with 20 cads. Psychedelic stuff like "I have a balanced 15-17 so I open 1♦ which is my longest suit as I have six clubs" (chatGPT on the hand with seven hearts). But at least they didn't make any insufficient bids.

 

I am disappointed with Bard who has demonstrated reasonable knowledge of the SAYC bidding system on other deals. Not this time, although at least Bard bids diamonds when he has diamonds and hearts when he has hearts, while chatGPT bids fairly randomly. chatGPT is still to show any sign of life bridge-wise, as far as I have observed.

[pescetom]

Yesterday one of our beginners (after 6 months of teaching and practice) was surprised that spades did not beat hearts in a trick of NT. So there is some hope for Bard.

[helene_t]

When comparing Bard and chatGPT, my impression is that Bard has better reasoning skills but poorer language skills. For example, Bard will get basic VAT calculations right (chatGPT doesn't). But Bard is sensible to the exact way the question is phrased and is more likely to misunderstand questions than chatGPT is.

 

So whether you write the 10 as "10" or as "T" makes no difference to chatGPT, while Bard gives different answers. I think that it is the "10" which it interprets correctly but I am not 100% sure.

 

It is possible that Bard would bid better if I took more care, maybe refering to the other players as N, S, and E instead of LHO, RHO and partner. Or some such.

 

Bart can generally determine opening bids for SAYC but when I ask it to bid using Acol it confuses the notrump range. Maybe it uses a mixture of Dutch Acol and English Acol sources.

 

When you ask Bard to tell you a joke about for example how many bridge players are needed to change a lightbulb, it will tell you a boring joke and then elaborate on why the joke is funny.

[LBengtsson]

Artificial Intelligence will never beat Common Sense. Period.

[barmar]

What would possibly make you think that a predictive language model would know anything about bridge bidding? All it knows how to do is find patterns of words in its database, and put these patterns together to produce its texts.

 

So unless there's a web page that includes your hand diagrams and then a bidding diagram, where would it come up with bidding for this?

 

Repeat after me: ChatGPT doesn't actually know anything. It's just repeating things it has heard.

[pilowsky]

Repeat after me: ChatGPT doesn't actually know anything. It's just repeating things it has heard.

 

So quite similar to most bridge players in a way.

[helene_t]

What would possibly make you think that a predictive language model would know anything about bridge bidding?

 

chatGPT can play chess and it can make sample size calculations for clinical trials. Basically, it can predict what a popular answer on Redit would be to a wide variety of questions.

 

When chatGPT plays chess, it will occasionaly move pieces that don't exist, which confirms that the rules and objectives of the game haven't been programmed explicitly, but apparently there are enough chess discussions in its inputs to allow it to play decent chess for the most part, i.e. predicting what a redit user would answer to just about any question about what the best move would be in a given situation.

 

I don't see any reason why it shouldn't be able to play bridge also. But of course bridge is more challenging. What bid a forum user might recommend in a given situation depends on partnership agreements and the opponents' system. And maybe bridge also has less standardized input formats than chess has, or simply less data.

[barmar]

If there were microphones distributed throughout all the bridge tournament venues, recording all the "you hold...." conversations and uploading them to web sites, ChatGPT *might* get enough information to create sensible bridge auctions.

 

I'll bet its chess answers get progressively less sensible as the game progresses. There are only a few openings, so it will easily learn these just as human chess players do.

 

This is similar to what it's like programming bridge robots. The basic opening bids are easy to program, but it gets much more complicated after 2 or 3 rounds of bidding because there are so many combinations of previous bids. Bridge programs know how to merge the information implied by each bid to create a general representation of the other players' hands, and they use this. But a generative language model like ChatGPT doesn't do that, it's just matching patterns of text.

[pilowsky]

[bobtehnoob]

Have you seen CGPT try to play chess? It's moves make sense for the first handful of them, but it's just reciting common move responses to its opponents moves, rather than legal moves based on the board state, because CGPT does not know the rules of chess so it cannot compute the board state.

[smerriman]

Have you seen CGPT try to play chess? It's moves make sense for the first handful of them, but it's just reciting common move responses to its opponents moves, rather than legal moves based on the board state, because CGPT does not know the rules of chess so it cannot compute the board state.

You're probably thinking of v3, and not v4. It's still not very good, but it's able to play complete games of chess.