Interpretable Function Approximation with Gaussian Processes in Value-Based Model-Free Reinforcement Learning

Matthijs van der Lende; Matthia Sabatelli; Juan Cardenas-Cartagena

Interpretable Function Approximation with Gaussian Processes in Value-Based Model-Free Reinforcement Learning

Matthijs van der Lende
, Matthia Sabatelli
, Juan Cardenas-Cartagena^*

^*Corresponding author for this work

Artificial Intelligence

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

51 Downloads (Pure)

Abstract

Estimating value functions in Reinforcement Learning (RL) for continuous spaces is challenging. While traditional function approximators, such as linear models, offer interpretability, they are limited in their representation capabilities. In contrast, Deep Neural Networks (DNN) can model more complex functions but are less interpretable. Gaussian Process (GP) models bridge this gap by offering interpretable uncertainty estimates while modelling complex nonlinear functions. This work introduces a Bayesian nonparametric framework using GPs, including Sparse Variational (SVGP) and Deep GPs (DGP), for off-policy and on-policy learning. Results on popular classic control environments show that SVGPs/DGPs outperform linear models but converge slower than their DNN counterparts.

Original language	English
Title of host publication	Proceedings of the Northern Lights Deep Learning Workshop 2025
Publisher	ML Research Press
Pages	141-154
Number of pages	14
Publication status	Published - 2025
Event	6th Northern Lights Deep Learning Conference, NLDL 2025 - Tromso, Norway Duration: 7-Jan-2025 → 9-Jan-2025

Publication series

Name	Proceedings of Machine Learning Research
Volume	265
ISSN (Print)	2640-3498

Conference

Conference	6th Northern Lights Deep Learning Conference, NLDL 2025
Country/Territory	Norway
City	Tromso
Period	07/01/2025 → 09/01/2025

Access to Document

Interpretable Function Approximation with Gaussian Processes in Value-Based Model-Free Reinforcement LearningFinal publisher's version, 1.12 MBLicence: CC BY

https://raw.githubusercontent.com/mlresearch/v265/main/assets/lende25a/lende25a.pdfLicence: CC BY

Handle.net

Persistent link

Cite this

van der Lende, M., Sabatelli, M., & Cardenas-Cartagena, J. (2025). Interpretable Function Approximation with Gaussian Processes in Value-Based Model-Free Reinforcement Learning. In Proceedings of the Northern Lights Deep Learning Workshop 2025 (pp. 141-154). (Proceedings of Machine Learning Research; Vol. 265). ML Research Press. https://raw.githubusercontent.com/mlresearch/v265/main/assets/lende25a/lende25a.pdf

@inproceedings{9a8d0b30887b42d093a8596418846a1d,

title = "Interpretable Function Approximation with Gaussian Processes in Value-Based Model-Free Reinforcement Learning",

abstract = "Estimating value functions in Reinforcement Learning (RL) for continuous spaces is challenging. While traditional function approximators, such as linear models, offer interpretability, they are limited in their representation capabilities. In contrast, Deep Neural Networks (DNN) can model more complex functions but are less interpretable. Gaussian Process (GP) models bridge this gap by offering interpretable uncertainty estimates while modelling complex nonlinear functions. This work introduces a Bayesian nonparametric framework using GPs, including Sparse Variational (SVGP) and Deep GPs (DGP), for off-policy and on-policy learning. Results on popular classic control environments show that SVGPs/DGPs outperform linear models but converge slower than their DNN counterparts.",

author = "\{van der Lende\}, Matthijs and Matthia Sabatelli and Juan Cardenas-Cartagena",

year = "2025",

language = "English",

series = "Proceedings of Machine Learning Research",

publisher = "ML Research Press",

pages = "141--154",

booktitle = "Proceedings of the Northern Lights Deep Learning Workshop 2025",

}

van der Lende, M, Sabatelli, M & Cardenas-Cartagena, J 2025, Interpretable Function Approximation with Gaussian Processes in Value-Based Model-Free Reinforcement Learning. in Proceedings of the Northern Lights Deep Learning Workshop 2025. Proceedings of Machine Learning Research, vol. 265, ML Research Press, pp. 141-154, 6th Northern Lights Deep Learning Conference, NLDL 2025, Tromso, Norway, 07/01/2025. <https://raw.githubusercontent.com/mlresearch/v265/main/assets/lende25a/lende25a.pdf>

Interpretable Function Approximation with Gaussian Processes in Value-Based Model-Free Reinforcement Learning. / van der Lende, Matthijs; Sabatelli, Matthia ; Cardenas-Cartagena, Juan.
Proceedings of the Northern Lights Deep Learning Workshop 2025. ML Research Press, 2025. p. 141-154 (Proceedings of Machine Learning Research; Vol. 265).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Interpretable Function Approximation with Gaussian Processes in Value-Based Model-Free Reinforcement Learning

AU - van der Lende, Matthijs

AU - Sabatelli, Matthia

AU - Cardenas-Cartagena, Juan

PY - 2025

Y1 - 2025

N2 - Estimating value functions in Reinforcement Learning (RL) for continuous spaces is challenging. While traditional function approximators, such as linear models, offer interpretability, they are limited in their representation capabilities. In contrast, Deep Neural Networks (DNN) can model more complex functions but are less interpretable. Gaussian Process (GP) models bridge this gap by offering interpretable uncertainty estimates while modelling complex nonlinear functions. This work introduces a Bayesian nonparametric framework using GPs, including Sparse Variational (SVGP) and Deep GPs (DGP), for off-policy and on-policy learning. Results on popular classic control environments show that SVGPs/DGPs outperform linear models but converge slower than their DNN counterparts.

AB - Estimating value functions in Reinforcement Learning (RL) for continuous spaces is challenging. While traditional function approximators, such as linear models, offer interpretability, they are limited in their representation capabilities. In contrast, Deep Neural Networks (DNN) can model more complex functions but are less interpretable. Gaussian Process (GP) models bridge this gap by offering interpretable uncertainty estimates while modelling complex nonlinear functions. This work introduces a Bayesian nonparametric framework using GPs, including Sparse Variational (SVGP) and Deep GPs (DGP), for off-policy and on-policy learning. Results on popular classic control environments show that SVGPs/DGPs outperform linear models but converge slower than their DNN counterparts.

UR - https://www.scopus.com/pages/publications/85219131527

M3 - Conference contribution

AN - SCOPUS:85219131527

T3 - Proceedings of Machine Learning Research

SP - 141

EP - 154

BT - Proceedings of the Northern Lights Deep Learning Workshop 2025

PB - ML Research Press

T2 - 6th Northern Lights Deep Learning Conference, NLDL 2025

Y2 - 7 January 2025 through 9 January 2025

ER -

Interpretable Function Approximation with Gaussian Processes in Value-Based Model-Free Reinforcement Learning

Abstract

Publication series

Conference

Access to Document

Handle.net

Other files and links

Fingerprint

Cite this