Publication
GRAIL: Autonomous Concept Grounding for Neuro-Symbolic Reinforcement Learning
Hikaru Shindo; Henri Rößler; Quentin Delfosse; Kristian Kersting
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2604.16871, Pages 1-23, arXiv, 2026.
Abstract
Neuro-symbolic Reinforcement Learning (NeSy-RL) combines symbolic reasoning with gradient-
based optimization to achieve interpretable and generalizable policies. Relational concepts—such
as “left of” or “close by”—serve as foundational building blocks that structure how agents per-
ceive and act. However, conventional approaches require human experts to manually define
these concepts, limiting adaptability since concept semantics vary across environments. We pro-
pose GRAIL (Grounding Relational Agents through Interactive Learning), a framework that au-
tonomously grounds relational concepts through environmental interaction. GRAIL leverages large
language models (LLMs) to provide generic concept representations as weak supervision, then re-
fines them to capture environment-specific semantics. This approach addresses both sparse reward
signals and concept misalignment prevalent in underdetermined environments. Experiments on
the Atari games Kangaroo, Seaquest, and Skiing demonstrate that GRAIL matches or outperforms
agents with manually crafted concepts in simplified settings, and reveals informative trade-offs be-
tween reward maximization and high-level goal completion in the full environment.