Learning to Play Pac-Xon with Q-Learning and Two Double Q-Learning Variants

Jits Schilperoort, Ivar Mak, Madalina M. Drugan, Marco Wiering

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

10 Citations (Scopus)
273 Downloads (Pure)


Pac-Xon is an arcade video game in which the player tries to fill a level space by conquering blocks while being threatened by enemies. In this paper it is investigated
whether a reinforcement learning (RL) agent can successfully learn to play this game. The RL agent consists of a multilayer perceptron (MLP) that uses a feature representation of the game state through input variables and gives Q-values for
each possible action as output. For training the agent, the use of Q-learning is compared to two double Q-learning variants, the original algorithm and a novel variant. Furthermore, we have set up an alternative reward function which presents higher rewards towards the end of a level to try to increase the performance of
the algorithms. The results show that all algorithms can be used to successfully learn to play Pac-Xon. Furthermore both double Q-learning variants obtain significantly higher performances than Q-learning and the progressive reward function does not yield better results than the regular reward function.
Original languageEnglish
Title of host publication2018 IEEE Symposium Series on Computational Intelligence (SSCI)
Number of pages8
ISBN (Print)978-1-5386-9276-9
Publication statusPublished - Nov-2018
Event 2018 IEEE Symposium Series on Computational Intelligence - Bengaluru, India
Duration: 18-Nov-201821-Nov-2018


Conference 2018 IEEE Symposium Series on Computational Intelligence

Cite this