Publikation
From Weights to Activations: Is Steering the Next Frontier of Adaptation?
Simon Ostermann; Daniil Gurgurov; Tanja Bäumel; Michael Aloys Hedderich; Sebastian Lapuschkin; Wojciech Samek; Vera Schmitt
In: Maria Liakata; Viviane P. Moreira; Jiajun Zhang; David Jurgens (Hrsg.). Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Annual Meeting of the Association for Computational Linguistics (ACL-2026), San Diego, California, United States, Pages 29854-29879, ISBN 979-8-89176-390-6, Association for Computational Linguistics, 2026.
Zusammenfassung
Post-training adaptation of large language models is commonly achieved through parameter updates or input based methods such as fine-tuning, parameter-efficient adaptation, and prompting. In parallel, a growing body of work modifies internal activations at inference time to influence model behavior, an approach known as *steering*. Despite increasing use, steering is rarely analyzed within the same conceptual framework as established adaptation methods.In this work, we argue that steering should be regarded as a form of model adaptation. We introduce a set of functional criteria for adaptation methods and use them to compare steering approaches with classical alternatives. This analysis positions steering as a distinct adaptation paradigm based on targeted interventions in activation space, enabling local and reversible behavioral change without parameter updates. The resulting framing clarifies how steering relates to existing methods, motivating a unified taxonomy for model adaptation.
Projekte
- TRAILS - Vertrauenswürdige und integrative Maschinen
- ToCertain - Towards a Centre for European Research in Trusted Artificial Intelligence