This article originally appeared in the November 1999 issue of GammOnLine.|
Thank you to Kit Woolsey for his kind permission to reproduce it here.
When I first went through games played by TD-Gammon (the first neural
network backgammon playing program), I looked for unusual plays which were
different from what I would have done. One recurring theme I noticed was
that TD-Gammon would often break up its board in a holding game for no
apparent reason. For example:
What could be more obvious than 13/8, 6/5, right? Yet TD-Gammon played the seemingly inexplicable 13/8, 3/2! Initially I assumed it was one of those quirks with the neural network programs. I knew that in simple positions the programs often make strange-looking (and wrong) moves because their training doesn't give them enough information to work out the proper principles. Human experts will usually outplay the bots in these sort of positions when it comes to small technical differences. I joked about this with Gerry Tesauro, the inventor of TD-Gammon, and called it the TD weird play.
A little later when Gerry had programmed TD-Gammon to roll out positions, I had him roll a couple of these types of plays out just to make sure. Much to my surprise, the TD weird plays consistently outperformed the normal plays in the rollout. This was something else. While the program might have some strange ideas, it generally played well and it was hard to see how it could badly screw up the rollouts for a position like this. Even if there were some kind of bias in the program's playing of holding games, the positions after the two plays in question are so similar that the bias would certainly apply to either play so the rollout results would be accurate. The absolute value might not be correct if the bot is misplaying one side or the other, but the relative results between the two plays almost has to be accurate.
Offhand, it doesn't appear as though it could make much difference even if one play were superior. Rollouts show that the difference can be more significant than we might think. I had Snowie roll the two plays out (1-ply, cubeless, 3888 times each with the same dice), and the results were:
13/8, 3/2 +.410 13/8, 6/5 +.362This is quite a significant difference for what seems to be an almost meaningless ace. Granted the sample size isn't huge, but with the duplicate dice and the similarity of the positions it appears quite clear that the TD weird play is noticeably superior. This is not an isolated case. I found this sort of play consistently won in the rollouts. What is going on?
Let's look ahead and see how the game is likely to continue. It is clear that Blue isn't going to be hitting any blots for quite some time. White is behind in the race, so even if White had a free ride to clear the bar point he would rather remain there to keep contact. Only if White rolls large doubles will White even consider leaving the anchor. Therefore, there is no urgent necessity to have the inner board points closed for the next few rolls.
If Blue rolls doubles, Blue will of course clear the midpoint and probably win the race. It won't matter much which ace Blue has played, but splitting to the two point does diversify Blue's checkers for the bearoff so that can't be bad. Similarly if White rolls large doubles White can run off the anchor and turn the game into a straight race. Again, splitting to the two point is fine. The more interesting scenario occurs when neither side rolls doubles for several rolls. Blue will have to bring his checkers in as best he can. His first priority will be to play safe, but he also wants to waste as little as possible. One large doubles by White will turn the game into a straight race, and if Blue has wasted too much his pip lead will go down the drain. Blue's goal is to fill his inner board smoothly, avoiding gaps on the lower points. This will be best both for the race and if White gets squeezed off the anchor and Blue gets a shot. What Blue wants to avoid is piling a bunch of checkers on one point while leaving another point empty.
Now that we see what Blue is trying to accomplish, it becomes clearer why 13/8, 3/2 is superior to 13/8, 6/5. First of all, having the spare on the six point is a definite plus, since that increases Blue's flexibility. More important, the split to the two point improves Blue's chances of smoothing out his position. After the split, if a Blue checker lands on either the three point or the two point it looks fine. If Blue plays 13/8, 6/5, then he needs to have a checker land precisely on the two point for the smoothing process to work out.
As an illustration, suppose White rolls 6-4 playing 13/7, 13/9, and Blue's next roll is 6-4. Naturally Blue will bring a checker in from the midpoint. Blue wants to clear the midpoint as soon as possible, and he wants to keeps spares on the eight point to handle awkward numbers. If Blue had played 13/8, 6/5 the position after he now plays 13/3 will look like this:
It's starting to look a bit ugly. Three checkers on the three point, and the ace and two points still bare. That third checker on the three point belongs on the two point, but it won't get there unless Blue rolls an ace. The key is the Blue did roll that ace his previous turn, and that is when the checker should have been moved to the two point. Had Blue played properly, the position would instead look like:
Isn't that much nicer? Now if Blue's next roll is something like 5-3, he can play it decently without tying himself in knots. In the position with three checkers on the three point and the six point stripped, a 5-3 roll plays awfully.
I'll grant that I chose a particularly awkward sequence of rolls for Blue to illustrate my point. However, these awkward sequences are the ones which matter. If Blue rolls a bunch of 2-1's it won't make any difference how he played previous rolls, since he will have a chance to reshuffle his checkers to where they belong. It is the large rolls such as 6-4 which create the awkwardness if you have failed to plan ahead and place your checkers properly.
Blue should play 8/1, not 8/2 5/4. This is a fairly important play. The problem is that if Blue rolls a six next, he will have to clear his eight point in order to play safe. If Blue has played 8/2, 5/4 then this six will force him to put a third checker on the two point, which is awful from flexibility, racing, and board building considerations. After playing 8/1 Blue will be putting a second checker on the two point, which is fine.
The main thing you have to watch out for when considering this sort of board breaking play is that you won't be getting a shot in the next couple of rolls, so you will have time to put the board back together. If you are ahead in the race this is almost always the case, since your opponent will be hanging back in order to retain as much contact as possible. If you are behind in the race, however, breaking your board may give your opponent a chance to pay now and make a run for it. For example:
Since White has three checkers on the midpoint, there is no way Blue will be getting a shot next roll. Even if Blue messes up his board, White will not be moving two checkers from the midpoint and leaving multiple shots. Therefore, Blue can and should play 8/6, 4/3. This will make it easier for him to complete both the four and three points in his next couple of rolls, which is what he is trying to do. There is little danger in temporarily unmaking the four point, since Blue has so many rolls to remake it. If Blue intead plays 9/6, he will have to roll more perfectly to make the three point. To see the difference, imagine Blue's next roll being 6-1 and look how the roll plays from both positions.
A small modification of the position can change things. Consider the following:
Same position except that now White has only two checkers on the midpoint. The difference is that Blue may be getting a shot next roll. If White rolls something like 5-3, White may choose to pay now and play 13/8, 13/10. This payment is obviously more attractive if Blue's board is a mess. Therefore Blue should play the normal 9/6 rather than the fancy board breaking play.
Most readers are familiar with point switching plays in blitz situations where you switch to make a lower point in order to have more builders aimed at the slotted point. If you can't move the builders to the point, move the point to the builders. The same principle can be applied in holding games, using board breaking plays.
I believe Blue's best play is the odd-looking 13/9, 5/3. White definitely won't be leaving a shot next roll, so Blue can afford to break his board temporarily. The loss of the five point is meaningless, since Blue gets it back with any roll but 5-2. The key is that Blue slots the three point, which is the next point he wants to make. If Blue makes another play the three point remains unslotted, and while Blue will usually be able to slot and make it before White leaves a shot occasionally the dice won't cooperate. In essence by playing 5/3 Blue activates the builder on the 11 point without having to move the checker. Naturally if White had only two checkers on the midpoint Blue could not afford the luxury of breaking his board, since White might volunteer a shot and clear the midpoint.
Granted the gains from these board breaking plays is small, but every little bit helps. Our goal is to maximize our equity all the time. Also this sort of position is very common, so the cumulative gains over many games can be significant. The next time you are building your board while playing a holding game and it seems as though your plays are awkward and you are tied up in knots, go back and see if you missed an opportunity for a profitable board breaking play. It is likely you will find those awkward rolls would have played much smoother had you prepared for them appropriately.