Publikation
Directional Constraints for Efficient Exploration in Safe Reinforcement Learning
Paolo Magliano; Puze Liu; Jan Peters; Davide Tateo; Raffaello Camoriano
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2607.12784, Pages 1-8, arXiv, 2026.
Zusammenfassung
Reinforcement Learning has revolutionized the
landscape of robotic research, allowing robust learning of
complex robotic skills in simulation. However, real-world de-
ployment in open-ended environments requires strong safety
guarantees to prevent dangerous or harmful behaviors. Safe
Reinforcement Learning methods address this requirement
by enforcing safety constraints. Nevertheless, learning under
constraints often reduces learning speed and could lead to
suboptimal task performance, as the agent must solve a
more complex constrained optimization problem compared
to unconstrained settings. To tackle this issue, in this work,
we propose an extension of the ATACOM framework, a
state-of-the-art reliable safety layer that can be integrated
with existing Reinforcement Learning algorithms to enforce
constraints derived from prior knowledge of the system or
learned directly from data. Our proposed method, named
ATACOM Directional Constraints (ATACOM-DC), significantly
improves the safety-performance trade-off by introducing di-
rectional constraints that distinguish between actions approach-
ing and moving away from constraint boundaries, activating
constraint enforcement only when necessary. We evaluate our
method across a range of challenging robotic control tasks
in simulation, analyzing both constraint-violation costs and
achieved task performance. Code and additional material at
https://atacom-dc.robot-learning.net