Learning to Walk With Deep Reinforcement Learning: Forward Dynamic Simulation of a Physics-Based Musculoskeletal Model of an Osseointegrated Transfemoral Amputee: Forward Dynamic Simulation of a Physics-Based Musculoskeletal Model of an Osseointegrated Transfemoral Amputee

Brown Ogum, Lambert Schomaker, Raffaella Carloni*

*Bijbehorende auteur voor dit werk

OnderzoeksoutputAcademicpeer review

18 Downloads (Pure)

Samenvatting

This paper leverages the OpenSim physicsbased simulation environment for the forward dynamic
simulation of an osseointegrated transfemoral amputee
musculoskeletal model, wearing a generic prosthesis.
A deep reinforcement learning architecture, which combines the proximal policy optimization algorithm with
imitation learning, is designed to enable the model to walk
by using three different observation states. The first is a
complete state that includes the agent’s kinematics, ground
reaction forces, and muscle data; the second is a reduced
state that only includes the kinematics and ground reaction
forces; the third is an augmented state that combines the
kinematics and ground reaction forces with a prediction
of the muscle data generated by a fully-connected feedforward neural network. The empirical results demonstrate
that the model trained with the augmented observation
state can achieve walking patterns with rewards and gait
symmetry ratings comparable to those of the model trained
with the complete observation state, while there are no
symmetric walking patterns when using the reduced observation state. This paper shows the importance of including
muscle data in a deep reinforcement learning architecture for the forward dynamic simulation of musculoskeletal
models of transfemoral amputees.
Originele taal-2English
Pagina's (van-tot)431-441
Aantal pagina's12
TijdschriftIEEE Transactions on Neural Systems and Rehabilitation Engineering
Volume32
DOI's
StatusPublished - 10-jan.-2024

Citeer dit