Publication
Reliable Fusion of Conflicting Experts
Pranuthi Tenali; Sahil Sidheekh; Saurabh Mathur; Vijayalakshmi Saravanan; Erik Blasch; Kristian Kersting; Sriraam Natarajan
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2609.27913, Pages 1-9, arXiv, 2026.
Abstract
e study the problem of aggregating opinions from multiple
black-box experts in noisy, conflict-prone settings where expert reliability
varies across inputs. Static aggregation methods, such as majority voting,
fail to capture this variability and often yield unreliable outcomes under
disagreement. We propose a tractable, probabilistic-circuit-based fusion
framework that dynamically combines expert responses using context-
specific credibility estimates, enabling principled and reliable reasoning.
The framework is agnostic to the underlying experts and does not require
access to their internal representations or any retraining. We empirically
validate our approach on multiple-choice question answering tasks us-
ing multiple LLMs as experts, comparing against individual models and
static ensemble baselines. Our method consistently improves predictive
performance and produces more reliable decisions under conflict, high-
lighting the effectiveness of context-aware credibility modeling for robust
multi-expert fusion.