Publication
Task-Error Residual Learning for Real-Robot Five-Ball Juggling
Kai Ploeger; Jan Peters
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2606.16978, Pages 1-16, arXiv, 2026.
Abstract
For residual learning that refines existing behavior, sample
efficiency depends on two things: how much information each rollout
returns, and how efficiently the learner uses that information. Reinforce-
ment learning’s standard scalar reward carries far less information than
the directional task error that defines the task. Random exploration fur-
ther discards whatever information each rollout returns. Through resid-
ual learning with directional task-error supervision and a task error
model that drives sample selection, we achieve stable three-, four-, and
five-ball juggling on anthropomorphic Barrett WAM arms. Despite plan-
ning and controlling through a simple, idealized stack, the system con-
verges from the second attempt. The first attempt drops, after which task
error decreases monotonically without further failures. In comparison,
five-ball juggling typically takes humans years of practice. We compare
residual learners across two ternary axes, the directional information in
the learning feedback and the commitment of the analytic prior, span-
ning Newton-style Jacobian updates, Composite Bayesian Optimization,
and stochastic search methods. Both axes prove necessary: neither direc-
tional feedback nor an informative prior suffices alone, and the simplest
method that combines them, a fixed-Jacobian Newton update, is the
most reliable. The learned residual tolerates substantial prior misalign-
ment and degraded joint tracking, affecting mainly convergence speed.
The bottleneck for residual learning on real robots is therefore the in-
formation content of the supervision signal and how the learner uses it,
not the accuracy of the surrounding stack.