## Samenvatting

Balancing the ratio between exploration and exploitation is an important problem in reinforcement learning. This paper evaluates four different exploration strategies combined with Q-learning using random stochastic mazes to investigate their performances. We will compare: UCB-1, softmax, epsilon-greedy, and pursuit. For this purpose we adapted the UCB-1 and pursuit strategies to be used in the Q-learning algorithm. The mazes consist of a single optimal goal state and two suboptimal goal states that lie closer to the starting position of the agent, which makes efficient exploration an important part of the learning agent. Furthermore, we evaluate two different kinds of reward functions, a normalized one with rewards between 0 and 1, and an unnormalized reward function that penalizes the agent for each step with a negative reward. We have performed an extensive grid-search to find the best parameters for each method and used the best parameters on novel randomly generated maze problems of different sizes.

The results show that softmax exploration outperforms the other strategies, although it is harder to tune its temperature parameter. The worst performing exploration strategy is epsilon-greedy.

The results show that softmax exploration outperforms the other strategies, although it is harder to tune its temperature parameter. The worst performing exploration strategy is epsilon-greedy.

Originele taal-2 | English |
---|---|

Titel | 2016 IEEE Symposium Series on Computational Intelligence, SSCI 2016 |

Uitgeverij | Institute of Electrical and Electronics Engineers Inc. |

Aantal pagina's | 8 |

ISBN van elektronische versie | 978-1-5090-4240-1 |

ISBN van geprinte versie | 978-1-5090-4241-8 |

DOI's | |

Status | Published - 9-feb-2017 |

Evenement | 2016 IEEE Symposium Series on Computational Intelligence, SSCI 2016 - Athens, Greece Duur: 6-dec-2016 → 9-dec-2016 |

### Publicatie series

Naam | 2016 IEEE Symposium Series on Computational Intelligence, SSCI 2016 |
---|

### Conference

Conference | 2016 IEEE Symposium Series on Computational Intelligence, SSCI 2016 |
---|---|

Land | Greece |

Stad | Athens |

Periode | 06/12/2016 → 09/12/2016 |