A Multimodal Dialogue Mashup for Medical Image Semantics

Daniel Sonntag, Manuel Möller

In: Proceedings of the International Conference on Intelligent User Interfaces. International Conference on Intelligent User Interfaces (IUI-2010) February 7-10 Hong Kong China 2/2010.


This paper presents a multimodal dialogue mashup where different users are involved in the use of different user interfaces for the annotation and retrieval of medical images. Our solution is a mashup that integrates a multimodal interface for speech-based annotation of medical images and dialogue-based image retrieval with a semantic image annotation tool for manual annotations on a desktop computer. A remote RDF repository connects the annotation and querying task into a common framework and serves as the semantic backend system for the advanced multimodal dialogue a radiologist can use.


Weitere Links


German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz