The goal of automotive IUI research at DFKI with its flagship project CARMINA is to lay the foundations of a new generation of context-aware multimodal interfaces for car passengers that support the interaction of the passengers with the car and mobile internet services, between the passengers inside the car, and between the passengers and the road environment.
The task-oriented dialog goes beyond the classical in-car command and control interface by enabling information seeking dialogs and by supporting the combination of speech and physical manipulations of tangible interfaces. The external context will be taken into account by supporting verbal references to landmarks visible in the surroundings of the car as well as by introducing a novel car2x paradigm. This paradigm enables on-demand information (user-initiated queries) on the basis of peer to peer (P2P) multi-hop networks between cars and the traffic infrastructure such as traffic signs and road side units. In addition, the multimodal user interface will not focus exclusively on the driver but will target the co-driver and the backseat passengers as well. Hence, the traditional dyadic dialog systems will be extended to multi-party systems that provide role-based access to available devices and services.
Research Questions
The nature of this future automotive scenario in conjunction with the new interaction aspects discussed above lead to the following basic research questions that are addressed for the first time in CARMINA:
- How can the system’s dialog behavior be synchronized with and adapted to a rapidly changing physical and spatial context induced by the car’s motion?
- How can the current multimodal fusion algorithms be generalized to support multi-party interaction and tangible interfaces where new approaches to mutual disambiguation of modalities are needed?
- How can the multimodal fission and media allocation algorithms be extended to cope with a situation that involves many LED indicators, multiple screens, and at least two audio output areas in premium cars of the future?
- However static the role allocation (driver, co-driver, passengers) during a single ride might be: How can the dialog be adapted to multiple role modalities or usage modes (e.g. chauffeur mode vs.family mode) ?
