Modern NLP models and LLMs have specific flaws, despite being highly performant: First, they are black boxes: Parameters of proprietary models are not accessible at all; and even non-proprietary models are largely opaque in the sense that it is unclear where exactly specific knowledge is encoded in potentially billions of parameters. Second, there is a tendency to always increase the size of LLMs and training data to improve performance, which is especially problematic for domains or languages with fewer resources.
The E&E group of DFKI’s Research Department Multilinguality and Language Technology works on transparent and efficient NLP models. Our objective is to make the parameters and behaviour of LLMs more explainable and understandable to both end users and researchers. We try to improve LLMs with regard to data consumption, e.g. for domains or languages where data is scarce, by using structured data, new learning techniques, or other modalities; and in terms of model size, e.g. for settings where powerful hardware is not available.
We are involved in Twinning projects, where we provide knowledge transfer both on research topics and project management to newly established research institutions across Europe. We are involved in European procurement projects focusing on language resources, such as the European Language Resource Coordination and the Language Data Space.
“Bridging the technology gap: Integrating Malta into European Research and Innovation efforts for AI-based language technologies”.
H2020-WIDESPREAD-2020-5 Grant Agreement No. 952194
Improving scientific excellence and creativity in combating disinformation with artificial intelligence and language technologies.
Consulting services to Gesellschaft für Internationale Zusammenarbeit (GIZ) on technical aspects of AI in international cooperation including natural language processing (NLP), training data and data access for FAIR Forward – Artificial Intelligence for All. GIZ Project No. 19.2010.7-003.00
In the XAINES project, the aim is not only to ensure explainability, but also to provide explanations (narratives). The central question is whether AI can explain in one sentence why it acted the way it did or whether it has to explain it interactively to the user. To clarify this, one of the focal points of the project is the exploration of narrative and interactive narratives, which are particularly suitable for humans to absorb knowledge in any form, in their application with AI systems.
Team Lead:
Dr. Simon Ostermann
simon.ostermann@dfki.de
Team Members:
Yusser al Ghussin
Tatiana Anikina
Tanja Bäumel
Róbert Belanec (Guest Researcher)
Rishu Kumar
Cennet Oguz
Stefania Racioppa
Soniya Vijayakumar
MSc Students and Research Assistants:
Konstantin Chernyshew
Daniil Gurgurov