Publication
DisCoMBO: Steering Expert-in-the-Loop Black Box Optimization via Distributional Conformance
Jonas Seng; Bennet Wittelsbach; Kristian Kersting
In: Computing Research Repository eprint Journal (CoRR), Vol. abs/2609.36472, Pages 1-41, arXiv, 2026.
Abstract
Sequential Model-Based Optimization (SMBO) traditionally relies on Bayesian
or ensembling surrogates for uncertainty quantification. While historically treated
as fully data-driven, SMBO increasingly integrates external domain expertise to
accelerate discovery. To overcome the opaque guidance and diminished integration
fidelity of standard acquisition re-weighting, Probabilistic Circuits (PCs) have
emerged as a generative surrogate alternative, enabling direct knowledge injec-
tion via conditional sampling. However, these generative routines lack the formal
exploration-exploitation semantics required for rigorous optimization. We intro-
duce the Distributional Conformance Score (DisCo), a novel metric that unifies
the flexibility and efficiency of PCs with a formal uncertainty framework. DisCo
provides a bounded, [0, 1]-normalized measure of model "surprise" that (1) recov-
ers properties comparable to kernel-based uncertainty known from, e.g., Gaussian
Processes, while maintaining linear-time inference, and (2) enables accurate assess-
ment of conformance of external knowledge w.r.t. model evidence. We then present
DisCoMBO, a framework leveraging these properties for robust, knowledge-aware
optimization. We prove that DisCoMBO is a zero-regret algorithm and demonstrate
its effectiveness across diverse benchmarks from AutoML, material optimization,
and wind park optimization.