This interview was originally written in German and translated into English.
Your research department benefits from your long-term experience at DFKI. How did it all start?
I actually began working at DFKI as a research assistant in 1998 in the Computational Linguistics and Language Processing research department. After that, I worked as a software developer for the company Xtramind, a DFKI spin-off. It was located in the then newly built DFKI building in Saarbrücken. Between 2002 and 2004, I worked in the Information Mining research department at DaimlerChrysler. I liked it very much, but there were not many opportunities to participate in scientific conferences, and the American partners were planning to rename and restructure the German research departments into the Advance Development departments. Fortunately, I was able to return to DFKI in 2004, to the Intelligent User Interfaces research department, led by Wolfgang Wahlster, whom I had met at the European Conference on AI (ECAI) in summer 2004. I have never regretted my decision to return to DFKI.
What happened then?
In 2016, I became a DFKI Research Fellow and then went on to become the director of the newly founded IML research department. My areas of expertise, then and now, are intelligent user interfaces, and information extraction from texts, images, and sensors.
What exactly does that mean?
On the one hand, it is about providing multimodal, adaptive, mobile, and sensor-equipped user interfaces. Given the increasing complexity in applications, such interfaces should adapt to humans and not the other way around. This applies to human-environment-interaction, human-robot-interaction, and personal smartphone assistants likewise. On the other hand, we want to achieve intuitive operability through the combination of semantic technologies and machine learning (for example Common Sense Reasoning and Deep Learning). As a result of this, we want to equip AI systems with more intelligence for everyday life in the domains of cognitive, sensomotoric, emotional, and social intelligence.
How was the name “IML” selected?
We have always understood intelligent user interfaces as an intersection of HCI (human-computer-interaction) and AI, and I want to continue this tradition. Therefore, we had to consider in detail which specialization we wanted to concentrate on in the research area and which subject we would represent within the fundamental research. The idea was to include as many data mining and machine learning methods from past projects as possible. Moreover, there was, and still is, an undeniable trend towards more Machine Learning (ML), and in my opinion, an undeniable convergence of AI and ML. Hence, the central topic - the intersection of HCI and ML- was quickly identified and named, consequently, as Interactive Machine Learning.
What does the research department focus on?
We design and implement intelligent algorithms and user interfaces that facilitate machine learning with the help of human interaction. We want computers to learn from humans by interacting with them using natural language and learning through observational data; thus, the focus is human-in-the-loop systems, sometimes the term “Machine Teaching” is used in this context.
It is important to emphasize that our research focuses on simplifying the process of teaching computers situations and intelligent behavior. I think in a few years, this will be of interest especially for small and medium-sized enterprises (SMEs). Human-centric AI-Research for SMEs will result in new medical, industrial, and generally more sustainable application systems that continuously improve over time, primarily based on human input.
It is particularly about processing the user’s input as efficiently as possible and at the same time, the new assistive systems can help the users save their time: to get more Human Time. On the one hand, it means adeptly motivating the end user to train an adaptive AI system, and on the other hand, effective assistance systems can be created which in turn save time for the respective expert. If thousands of “AI eyes” can look at numerous patients’ health records at the same time, it will surely be a new kind of medical decision-making basis.
In which application areas do you mainly want to utilize your research?
Our research in Applied AI mainly contributes to medical (ai-in-medicine.dfki.de) and industrial (smartfactories.dfki.de) applications. We also focus on the topic of sustainability and AI (https://cst.dfki.de/).
Can you give us an example?
Let us consider AI in medicine. The essential question in the context of transfer to applications is how IML systems and intelligent user interfaces can be made usable in medical applications.
In 2010, we, at DFKI, had already developed a speech dialog system for radiologists, and later digital dementia tests. Especially for such applications, as well as for AI in medicine in general, a few aspects will be of importance. Access to data and the integration in complex medical services, both in clinical and non-clinical contexts are crucial for a successful application and utilization of AI.
What is necessary for a successful transfer?
The amount of usable high-quality data must be increased significantly. In my opinion, the key to success will depend on a good strategy to acquire data and knowledge. Relatively, it is rather easy to replicate technologies, but in comparison and on a long-term basis, qualitative data must be built up arduously for processing with AI technologies.
In the long run, it would be advantageous for Germany to broaden and motivate the process of capturing basic patient data and other digital information regarding medical conditions. There are already prototype applications that have been realized by means of high-quality data from a clinical study in the project Klinische Datenintelligenz (KDI, clinical data intelligence) funded by the Bundesministerium für Wirtschaft und Klimaschutz (BMWK).
Many similar decision-support systems would have already been possible if there were existing data partnerships between clinics and research institutes. Through such partnerships, the potential of using AI in medicine could be better harnessed. However, the analysis of malfunctions of clinical decision-support systems is indispensably necessary. The same applies to industrial applications and data, as well as for those applications regarding sustainability.
What is the connection between IML and sustainability?
I like the following statement from DFKI’s CEO Antonio Krüger, presented in the recent corporate magazine: “The term sustainability focuses on a modern industrial value-addition as a necessary prerequisite for a high standard of living, good jobs and education, climate protection, a circular economy, and social participation.” It shows the diversity that the topic sustainability holds, and its direct relation with the economy and businesses, especially with respect to the UN’s 17 Sustainability Development Goals. However, it can be implied that with AI competence not just single products are developed (in the industrial sector), but in broader terms, we are talking about the technical ability to build internationally relevant and competitive systems.
However, it is to be understood how this can initially be achieved within a single research department, on a small scale. As in the case of computational linguistics, from which language technology would result, I am interested in pursuing computational sustainability in the future, which will result in sustainability technologies in combination with IML.
What is the point of a mobile AI lab?
That is something we are working on at the moment. Following the first wave of digitization of smart factories, which led to making data available in digital and mobile forms as well as interpreting them contextually in real time, we are now focusing on sustainability: a mobile AI lab. It can provide Edge-Support and therefore has a reduced latency, for example in Virtual Reality applications and real-time image processing. At the same time, it facilitates data sovereignty. As a result, the complex anonymization of patient data, for example, is no longer needed.
And how can the mobile AI lab save resources?
With a mobile AI lab, we want to pave the way to a lean and resource-efficient AI. With this approach, high-quality data sets can be generated faster and a strong transfer into applications can be assured by eliminating the need to constantly retrain ML-models. Interestingly, this path towards resource efficiency, i.e., learning with small data and learning with user feedback, is a central topic of IML and directly addresses the sustainability goals of generating a lower carbon-footprint.
The current energy consumption of GPUs in the U.S. produces about half as much CO2 as the entire flight-traffic in the U.S. In the future, research should deal with more efficient and effective ML-methods, i.e., data, models, and algorithms. One interesting question is how human-in-the-loop-ML-systems can contribute to this. How often do they need to be retrained, or is there a way to create incremental, iterative systems? With human-in-the-loop and self-learning capabilities? With simultaneous resource efficiency and long-term autonomy? To answer these questions, the requirements of the local data infrastructures of the mobile AI lab will certainly need to be considered, and I think this will be an exciting assignment. I am curious to see for which organization we will first perform a test at their doorstep, without any issues of infrastructure or bandwidth.
Your research department has grown significantly over the past one and a half years. How would you describe your team, which is working in Oldenburg and Saarbrücken?
Our approach for the next five years is the development of techniques that allow computers to receive human feedback on their actions, and simultaneously gather data automatically, by means of eye tracking or image recognition. Currently, interdisciplinary teams are working in IML on this mission, whose members have a sound academic background in media informatics, medical informatics, computer science, computational linguistics, linguistics and cognitive science in general, mathematics, human-computer-interaction, and theoretical and practical machine learning. I am sure that there will be a need for additional competences in the future if we want to optimize the IML-process with the goal of making it easier and faster for machines to learn from humans. Our IML-Team already comes from 18 different countries (https://iml.dfki.de/news/increasing-diversity-within-the-iml-team/). I want to thank all our current and former employees, especially Alexander Prange, Michael Barz, and Hans-Jürgen Profitlich, who co-founded the IML research department.
Which research project do you consider to be among the most important ones currently?
Probably the technical design of Explainability (XAI) of the AI models against the background of interactive machine learning. In 2019, I had the idea of giving the title “Interactive Machine Learning and Explainable AI, two sides of the same coin” for a talk at the AAAI Fall Symposium in Washington. By now, the ideas are much more specific, and they are subjects of the prospective project No-IDLE in 2023, which will be funded by the German Federal Ministry of Education and Research (BMBF). Its content will include interactive deep learning: AI systems are supposed to give explanations to their interactively learned models, especially in the context of real use cases with real images.
At the same time, they are supposed to learn the correct concept via dialog-based explanations by the experts, who are using the systems. The interaction with the user emphasizes the significance of cognitive sciences and language technology, because future robust AI systems cannot be designed as pure ML-based black-box systems. In this context, I am also thinking about a spin-off. Together with the large DAX companies, the emerging AI ecosystem comprising of six centers of excellence for AI research in Germany can offer a suitable, reliable infrastructure. Along this way, the results of those research-intensive topics can be directly integrated and potentially developed as the products for key clients of respective companies and SMEs. Therefore, the institutional support of the BMBF will prospectively be even more crucial in joint ventures with the businesses during the third wave of AI.
About Daniel Sonntag:
Prof. Dr. Daniel Sonntag is the Director of the DFKI “Interactive Machine Learning” (IML) research department in Oldenburg and Saarbrücken. At Oldenburg University, he holds the Endowed Chair of “Applied Artificial Intelligence” in the Department of Computing Science. Besides Artificial Intelligence in medicine and industry, sustainability plays an important role, and it is the focus of the research field "Computational Sustainability & Technology" in Oldenburg. In addition, Prof. Dr. Sonntag is the editor-in-chief of the German journal "Künstliche Intelligenz" by the German Informatics