Publikation
Safety-Constrained Contextual Bandit for Dynamic Power Management
Yasin Esfandiari; Karan Rajshekar; Cicy Kuriakose Agnes; Hannah Stein; Sabine Janzen; Wolfgang Maaß
In: Proceedings of the 1st Workshop on Sustainability and Resource-Efficiency of Artificial Intelligence (SuRE 2026) at IJCAI 2026. Workshop on Sustainability and Resource-Efficiency of Artificial Intelligence (SuRE-2026), located at IJCAI-ECAI 2026, August 17, Bremen, Germany, CEUR Workshop Proceedings, CEUR-WS.org, 2026.
Zusammenfassung
Real time AI model serving workloads are characterized by bursty and variable request patterns, yet production GPUs commonly operate at their maximum frequency, wasting substantial energy whenever latency headroom exists. Reducing energy consumption without violating tail-latency SLOs is challenging because frequency scaling affects latency in highly nonlinear, workload-dependent ways. This paper presents HALO (Hierarchical Adaptive Latency-Oriented DVFS), a fully black-box power-management controller that uses probe p95 latency to classify the system into coarse safety zones, gates the set of admissible frequency actions per zone, and runs a contextual bandit within those constraints to adapt online, requiring no offline profiling, model instrumentation, or servingframework modifications. For LLM inference on the Azure LLM Inference Trace (Coding workload), HALO reduces total GPU energy by 39.4% on the primary model and up to 50.0% across model scales, while maintaining tail latency under the SLO. Applied to LLM pretraining, the same controller reduces energy with convergence unaffected, demonstrating that the approach generalizes across both inference and training workloads.