Skip to main content Skip to main navigation

Publication

Towards a Mapping for AI-Based Eye-Tracking Analytics in Multimedia Learning

Konstantinos Tsiakas; Francesca Zermiani; Deniz Iren; Nghia Duong-Trung; Michael Raschke; Roland Klemke; Milos Kravcik; Ladislao Salmerón; Halszka Jarodzka; Leen Catrysse
In: Joshua Weidlich; Irene-Angelica Chounta; Kairit Tammets; Tanya Nazaretsky; Bibeg Limbu; Fridolin Wild; Pablo Arnau-González (Hrsg.). Mindful TEL: Learning Technologies Shaped with Intention. European Conference on Technology Enhanced Learning (EC-TEL-2026), 21st European Conference on Technology Enhanced Learning, ECTEL 2026, September 14-18, Valencia, Spain, Lecture Notes in Computer Science (LNCS), Vol. 16905, ISBN 978-3-032-37978-8, Springer, Cham, 9/2026.

Abstract

Supporting students’ reading comprehension is demanding for teachers, as they have limited insight into the reading processes of each student while they are reading. Although eye-tracking and artificial intelligence (AI) approaches offer strong potential to generate such insights, they remain difficult to deploy and apply in actual classroom practice. A key limitation lies in the lack of a structured approach to translate gaze data into pedagogically sound, classroom-ready, actionable insights for teachers. In this paper, we address this gap with a proof-of-concept mapping that connects the Cognitive Theory of Multimedia Learning (CTML) processes—selecting, organizing, and integrating— and their associated eye-tracking metrics with specific AI methods and HCI-driven visualizations. In this ongoing work, we present an initial mapping derived from a literature review: we identify eye-tracking metrics associated with CTML processes and propose preliminary connections to AI methods and HCI-driven visualizations. Drawing on this mapping, we illustrate our approach with a concrete example from a PISA task. We argue that this structured approach is a necessary step toward designing more targeted applications, moving from gaze data to feedback that teachers can use. By grounding technical and design choices in CTML, this mapping aims to support the development of AI-enhanced tools that are technically sound and pedagogically relevant.

Projects