Publication
An intelligent man-machine interface - multi-robot control adapted for task engagement based on single-trial detectability of P300
Elsa Andrea Kirchner; Su-Kyoung Kim; Hendrik Wöhrle; Marc Tabie; Michael Maurus; Frank Kirchner
In: Frontiers in Human Neuroscience, Vol. 10, Page 291, Frontiers, 6/2016.
Abstract
Advanced man-machine interfaces (MMIs) are being developed for teleoperating robots at
remote and hardly accessible places. Such MMIs make use of a virtual environment and can
therefore make the operator immerse him-/herself into the environment of the robot. In this
paper, we present our developed MMI for multi-robot control. Our MMI can adapt to changes in
task load and task engagement online. Applying our approach of embedded Brain Reading we
improve user support and efficiency of interaction. The level of task engagement was inferred
from the single-trial detectability of P300-related brain activity that was naturally evoked during
interaction. With our approach no secondary task is needed to measure task load. It is based on
research results on the single-stimulus paradigm, distribution of brain resources and its effect
on the P300 event-related component. It further considers effects of the modulation caused by
a delayed reaction time on the P300 component evoked by complex responses to task-relevant
messages. We prove our concept using single-trial based machine learning analysis, analysis
of averaged event-related potentials and behavioral analysis. As main results we show (1) a
significant improvement of runtime needed to perform the interaction tasks compared to a setting
in which all subjects could easily perform the tasks. We show that (2) the single-trial detectability
of the event-related potential P300 can be used to measure the changes in task load and task
engagement during complex interaction while also being sensitive to the level of experience
of the operator and (3) can be used to adapt the MMI individually to the different needs of
users without increasing total workload. Our online adaptation of the proposed MMI is based
on a continuous supervision of the operator’s cognitive resources by means of embedded Brain
Reading. Operators with different qualifications or capabilities receive only as many tasks as
they can perform to avoid mental overload as well as mental underload.
Projects
- IMMI - IMMI - Intelligent Man-Machine Interface
- TransTerrA - Semi-autonomous cooperative exploration of planetary surfaces including the installation of a logistic chain as well as consideration of the terrestrial applicability of individual aspects