Publication
Co-Exploration and Co-Exploitation via Shared Structure in Multi-Task Bandits
Sumantrak Mukherjee; Serafima Lebedeva; Valentin Margraf; Jonas Hanselle; Kanta Yamaoka; Viktor Bengs; Stefan Konigorski; Eyke Hüllermeier; Sebastian Vollmer
In: Nineteenth European Workshop on Reinforcement Learning. European Workshop on Reinforcement Learning (EWRL-2026), 19th European Workshop on Reinforcement Learning, October 5-7, Lille, France, DFKI Research Reports (RR), EWRL (OpenReview.net), European Workshop on Reinforcement Learning, 10/2026.
Abstract
We propose COCO-BANDITS, a Bayesian framework for efficient exploration in contextual multi-task multi-armed bandits with partially observed context and latent reward dependencies across tasks and arms. Our approach learns a global joint distribution across tasks while retaining personalised inference, and distinguishes structural uncertainty from user-specific uncertainty due to incomplete context and limited history. Concretely, we approximate the joint distribution over tasks and rewards with a particle-based log-density Gaussian Process, enabling discovery of inter-arm and inter-task dependencies without parametric latent assumptions. We show theoretically that learning the meta-prior yields sublinear cumulative multitask regret under posterior contraction assumptions. Empirically, on synthetic benchmarks and a real-world-dataset-based environment, our method outperforms parametric baselines and shows that nonparametric latent modelling improves robustness under misspecification and complex heterogeneity.