Visual Query Construction for Cross-Modal Semantic Retrieval of Medical Information

Manuel Möller, Nitya Vyas, Michael Sintek, Sven Regel, Saikat Mukherjee

In: Proceedings of the Malaysian Joint Conference on Artificial Intelligence (MJCAI). Malaysian Joint Conference on Artificial Intelligence (MJCAI) July 14-16 Kuala Lumpur Malaysia 7/2009.


We present an application for semantic annotation and retrieval of medical images. It leverages the MEDICO ontology which covers formal background information from various biomedical ontologies such as the Foundational Model of Anatomy (FMA), terminologies like ICD-10 and RadLex and also includes various aspects of clinical procedures. By annotating available data semantically our system gains a number of features not available in existing keyword-based search engines. Search can be performed independent of the modality. Thus, x-ray images, Computed Tomography 3D volume data sets and medical records in text format can be searched for the same medical concepts in a unified manner. The work presented here focuses on our approach for an incremental generation of semantic queries. Our technique does not require prior knowledge about structure and available concepts in the ontology. Instead, it allows the user to construct complex queries by exploring the formal background knowledge from the ontologies and to create queries using formal concepts. In this context we also present our approach for the mapping of RadLex and FMA. It allows to combine the rich semantic modeling of the FMA for query expansion with the lightweight radiology-oriented RadLex hierarchy for the user interface.


Weitere Links

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz