The project aims to develop a technological platform for the scientific validation of phenotypes for psychiatric disorders based on multimodal inputs such as speech, video and biosignals from clinical social interactions. To this end, researchers will collect data from video recordings, conversations, but also brain or heart activity (EEG, ECG).
When evaluating speech data, a semantic and paralinguistic analysis is carried out, whereby phenomena such as speech rate, repetition, and sentence melody are examined.
The MePheSTO team merges this speech, image, and bio-data into a multimodal corpus. From the multitude of heterogeneous data, machine learning methods such as Deep Learning classify the behavioral patterns and assign them to a psychiatric disease pattern.
It is also being investigated to what extent passive mobile phone data such as location information or movement patterns can be used to measure and evaluate patients' behavior in everyday life. In this way, an impending relapse, e.g., a beginning manic phase in the case of a bipolar disorder, could possibly be detected or predicted at an early stage. The protection of personal data is taken into account by a project-specific GDPR-compliant catalog of technical and organizational measures (TOMs).
In MePheSTO, scientists from the fields of speech and dialogue analysis, computer vision, and machine learning from DFKI and INRIA work together in a research core group. In addition to the scientific objectives, the project partners aim to establish a multidisciplinary, Franco-German team of experts over the next three to five years. To this end, DFKI and INRIA laid the foundation stone in a Memorandum of Understanding on 22 January 2020, the first anniversary of the Aachen Treaty. MePheSTO builds on a line of research projects in the health sector that have been realized within the European funding structures EIT Health and EIT Digital.
Project information
Funding Organization: Federal Ministry of Education and Research
Volume: 3 million euros
Duration: 1.8. 2020 - 31.7. 2023
Partners
German Research Center for Artificial Intelligence - DFKI, Saarbrücken
Institut National de Recherche en Informatique et Automatique - INRIA, Nancy
Centre Hospitalier Universitaire de Nice (Prof. Dr Philippe Robert)
Saarland University Hospital (Prof. Dr. Matthias Riemenschneider)
Centre Psychothérapique de Nancy (Prof. Dr Vincent Laprevote)
Centre Hospitalier Montperrin, Aix-en-Provence (Dr Sophie Barthelemy)
University Clinic for Psychiatry and Psychotherapy at the Karl-Jaspers-Klinik, Oldenburg (Prof. Dr. René Hurlemann)
Further information
https://www.dfki.de/en/web/news/detail/News/inria-and-dfki-sign-memorandum-of-understanding-for-cooperation-in-ai/
www.inria.fr
https://eithealth.eu