Publikation
PABeLA: A System for Auditing Societal and Media-Regulatory Criteria in Large Language Models
Stefan Schaffer; Niko Kleer; Pascal Lessel; Michael Feld; Julien Bauer; Lea Schmelz; Ina Goedert
In: Mensch und Computer 2026 - Tagungsband. Mensch und Computer (MuC-2026), Gesellschaft für Informatik e.V. 2026.
Zusammenfassung
LLMs increasingly influence how people access information, form opinions, and interact with digital services. Their role in public discourse and the media ecosystem raises questions traditionally addressed by media regulators, including nondiscrimination, free political opinion formation, and compliance with societal and democratic norms. However, the models’ internal mechanisms remain inaccessible, requiring black-box testing to evaluate normative alignment. As a preliminary step, a structured normative catalogue was developed in collaboration with a SMA. Building on prior work that identified regulator-relevant evaluation criteria for Large Language Models, high-level values drawn from Article 5 of the German Constitution, the Medienstaatsvertrag (State Media Treaty), the Digital Services Act, the AI Act, and international AI ethics guidelines were translated into technically testable criteria. This demonstrator shows the first two implemented criteria: detection of discrimination and free political opinion formation.