Analyzing your own matches can indeed be a quick process if you just click through them and let your bot do all your thinking for you. But why settle for that, when you are so much smarter? Here is the method I recommend for letting your bot assist you in your ceaseless quest for knowledge, enlightenment, and the ability to squash your opponents like so many little ants:

The Basics

1. Record the match as you play it (electronically is fine). Feed it through JellyFish, Snowie, or any other bot you may have that lets you run matches through it conveniently. I will base the rest of this discussion on JellyFish 3.0, since that is the bot I use. From what I have seen of them, I'd expect the procedure for Snowie or any other advanced bot to be pretty similar.

2. Help your bot find your errors. Set its gotcha point at no more than .03, since on its first pass it may not otherwise catch as much. (I use .028.) If it flags something, verify twice to see if it "reverses on appeal". If it ends up preferring your play after all when it gets to Level 7, you can generally assume your play was OK and continue. But just to be sure, you may want to backtrack and get the entire "Top Ten" list for the decision, since there may have been a third choice it likes better than either play! Also note the trend-was the difference between the top two choices (or the bot's top pick and yours) rising or falling from Level 5 through Level 7? If the difference was rising as the bot looked further ahead, chances are your play was dead wrong. If it was declining sharply, your play may have actually been correct if the final difference was not large.

3. Make the bot tell you about the errors it didn't catch. As you go through the match, even if the bot doesn't flag a play, if you are unsure, get the Top Ten list and verify twice to see what Jelly thinks when it gets to use its maximum lookahead. If the bot's Top Ten list does not include a play you think it should have considered, make it do so by feeding it the position after that play, then telling it to evaluate from the opponent's point of view.

   You can instead have it evaluate more than 10 plays each time, but I choose 10 because then if I want to print the position, it all fits on one page. Limiting yourself to 10 plays also makes sense if you have a slow machine, because otherwise it could take quite a while for Jellyfish to verify from Level 6 to Level 7, and almost all the extra time would be spent evaluating nonsense plays.

4. Toss out the evaluation if you know from experience that your particular bot is clueless in the type of game in question. Backgames are still a big weakness in bot play (and in quite a lot of human play, too). There are also some situations where the bots will run from an anchor prematurely, though a rollout will usually set them straight in those cases.

5. As best you can, distinguish among careless errors, matters of style, minor nuance plays, and true concept errors. This last group is the one you need to worry most about! Concept errors tend to involve game plan selection, a fork in the road where you make a choice from which there will be no turning back. If the error is large (I consider >.70 equity large), I find it helpful to print out the Level 7 evaluation page, then review it 3-6 months later along with all the others I have been collecting.

Extra Credit

6. Run evaluations for the same play at double match point (assuming the play occurred at some other score in the actual match) and also as a money problem. These cases help you to see how gammons or the lack thereof affect the decision in question. For some match scores, especially ones involving 2-, 3-, or 4-away scores, I find it helpful to do the match equity calculations by hand. This goes double for cube decisions, so to speak.

7. If you feel it is appropriate and convenient, do both Level 5 cubeful and Level 6 cubeless rollouts of the top contending plays (and also whatever you did, if it didn't make the list and you still are not convinced it is with good reason) You definitely need to roll out long-horizon endgame problems, and any problem where the evaluated difference was relatively small. I have found that "reversals on appeal" are quite common at the .04-type level. In all cases, "spot check" a couple of key plays to be sure the bot is not making massive errors in the Level 5 or Level 6 rollouts. This can affect the results big time, especially if one side is much more difficult to play than the other. Also, be aware of known "glitches" in the software you use, and check to see if opportunities to make these signature errors are likely to arise in your rollouts. If so, it may be better to believe the Level 7 evaluation or just your own judgment rather than the rollout results.

Extra-Terrestrial Credit

8. If there is still a significant difference between the best play and the second choice after the rollouts, determine what features of the position are most important. Do this by changing one feature at a time and seeing what effect it has on the play selections.

9. Propose a general principle governing the type of play you have just studied. If possible, come up with one or more radically different positions where you think the decision should be similar, based on your new principle, because the key features are similar to those of the position you studied. Ask the bot if it agrees.

Isn't this all kind of time-consuming? Well, uh, yeah, it is. Now that FIBS player who thought my experience points weren't going up very fast knows why. But for me, part of the "experience" gained from playing is learning as much as possible from my games. Backgammon is so much more satisfying, at least to me, when instead of getting the mere intermittent reinforcement that the dice often provide, I also have a sense of real progress.