The INSYS project is concerned with the interpretability of learned models and the resulting possibilities for self-monitoring of complex robotic systems working with multimodal data. This will be accomplished with the development of novel approaches of XAI for multimodal robotic systems to better understand the analysis of correlations of cause, in this case input data, and effect, or model output, and make it more explainable. On this basis, general correlations within the data can also be analyzed without much prior knowledge. For robotic systems, this means that on the basis of the data generated by various sensors and their consistency checks, it is possible to recognize new situations, anomalies or malfunctions and thus to monitor the correct function during the execution of various tasks. At the same time, the information obtained can be used to show mission control the analyses of an automatic system in a comprehensible way so that it can intervene in the event of errors. In this respect, the monitoring process with regard to the consistency check can be divided into two rough levels can be divided into two levels of the entire processing chain, from sensing and acutatorics to deep, multimodal neural networks individually pursued by the partners. Here, the University of Bremen is pursuing explainable and interpretable multimodal neural networks at the model level where the focus is the verification of the output values of the learned models individually or in a composite according to the definition of environmental conditions. DFKI RIC pursues the implementation of monitoring and consistency checking of multimodal robotic systems on the sensor level where checking sensor values and/or simple features correspond to the currently expected behavior and learned expectations.
Partners
Universität Bremen
