Self-Play and Using an Expert to Learn to Play Backgammon with Temporal Difference Learning

Marco A. Wiering

Research output: Contribution to journalArticleAcademicpeer-review

580 Downloads (Pure)


A promising approach to learn to play board games is to use reinforcement learning algorithms that can learn a game position evaluation function. In this paper we examine and compare three different methods for generating training games: 1) Learning by self-play, 2) Learning by playing against an expert program, and 3) Learning from viewing experts play against each other. Although the third possibility generates high-quality games from the start compared to initial random games generated by self-play, the drawback is that the learning program is never allowed to test moves which it prefers. Since our expert program uses a similar evaluation function as the learning program, we also examine whether it is helpful to learn directly from the board evaluations given by the expert. We compared these methods using temporal difference methods with neural networks to learn the game of backgammon.
Original languageEnglish
Pages (from-to)57-68
Number of pages12
JournalJournal of Intelligent Learning Systems and Applications
Publication statusPublished - 2010


  • Reinforcement learning
  • Machine learning

Cite this