Publication
Tractable Bayesian Dynamics Priors from Differentiable Physics for Learning and Control
Joe Watson; Benedikt Hahner; Boris Belousov; Jan Peters
In: 40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA@40). IEEE International Conference on Robotics and Automation (ICRA-2024), September 23-26, Rotterdam, Netherlands, IEEE, 2024.
Abstract
Statistical model-based reinforcement learning
methods should enable efficient data-driven policy optimization
for robotic systems. However, the success of these approaches
relies on how well the learned dynamics model generalizes
outside of its training data distribution, which is difficult to
ensure in practice with ‘black-box’ models unless inductive
biases are incorporated. We demonstrate how a differentiable
simulation model can be used to synthesize a tractable Gaussian
process prior using the linearized Laplace approximation, a
principled approximate inference technique for Gaussian pos-
teriors. Using this statistical model as an inductive bias, we can
perform exploration and decision-making in an informed way
using posterior sampling, performing reinforcement learning
for control, and active learning for system identification. In the
case of simulation-to-reality mismatch, we empirically investi-
gate how this physics-informed prior is best used, comparing
architecture- and objective-based approaches.