Publication
Towards Classifying Mother Tongue from Infant Cries - Findings Substantiating Prenatal Learning Theory
Tim Polzehl; Tim Herzig; Friedrich Wicke; Kathleen Wermke; Razieh Khamsehashari; Michiko Dahlem; Sebastian Möller
In: Interspeech 2024. Conference in the Annual Series of Interspeech Events (INTERSPEECH-2024), September 1-5, Kos, Greece, Pages 4199-4203, ISCA, 2024.
Abstract
In this work we introduce automatic mother tongue classification based on infant cries. We use data of 63 German and Japanese healthy, term-born neonates, and model their cries with the help of data augmentation and Pre-trained Audio Neural Networks (PANNs), leveraging transfer learning methods suited to the very limited data at hand. Applying small models on top of PANNs we obtain F1 scores of 85% and above on a held-out test set. We conduct several experiments analyzing model reliability, all of which indicate the network focuses on the actual infant cries rather than on confounding factors. We visualize the network focus to adhere to pitch contour and harmonics thereof, rendering prosody central for our model prediction. Eventually, our models add a novel objectively obtained perspective to neonate crying analysis, while our results substantiate an extremely early vocal learning indication.
Projects
- Medinym - AI-based anonymization of personal patient data in clinical text and voice databases
- VERANDA - Trustworthy anonymization of sensitive patient data for remote consultations