Learning to Ascend Stairs and Ramps: Deep Reinforcement Learning for a Physics-Based Human Musculoskeletal Model

This paper proposes to use deep reinforcement learning to teach a physics-based human
musculoskeletal model to ascend stairs and ramps. The deep reinforcement learning architecture
employs the proximal policy optimization algorithm combined with imitation learning and is trained
with experimental data of a public dataset. The human model is developed in the open-source
simulation software OpenSim, together with two objects (i.e., the stairs and ramp) and the elastic
foundation contact dynamics. The model can learn to ascend stairs and ramps with muscle forces
comparable to healthy subjects and with a forward dynamics comparable to the experimental training
data, achieving an average correlation of 0.82 during stair ascent and of 0.58 during ramp ascent
across both the knee and ankle joints.
Keywords: deep reinforcement learning; computer simulation