Publication
A Unified Approach for Semantic-Based Multimodal Interaction
Markus Löckelt; Matthieu Deru; Christian H. Schulz; Simon Bergweiler; Tilman Becker; Norbert Reithinger
In: Wolfgang Wahlster; Hans-Joachim Grallert; Stefan Wess; Hermann Friedrich; Thomas Widenka (Hrsg.). Towards the Internet of Services: The THESEUS Research Program. Pages 131-144, Cognitive Technologies, ISBN 978-3-319-06754-4, Springer, 10/2014.
Abstract
This chapter gives an overview of the basic dialogue shell for multimodal
interaction that was created as the foundation for the software components developed
in the sub-tasks in the CTC work package 4 of the Theseus research programme.
It explains the basic setup of WP4 systems for specific use cases. The
general processing paradigm for interaction in our framework using Semantic Interface
Elements as well as several dialogue phenomena handled by the dialogue shell
are described. Finally, a selection of different systems that were built for a range of
applications is presented briefly. They offer access to quite varied use-case scenarios
featuring multimodal input and output, including gestures, multi-touch, spoken
input and output, and interactions across mobile and stationary devices.