Publikation
Real-Time Feedback on Reader’s Engagement and Emotion Estimated by Eye-Tracking and Physiological Sensing
Akshay Palimar Pai; Jayasankar Santhosh; Shoya Ishimaru
In: Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers. International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp-2022), International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers, Association for Computing Machinery, New York, NY, United States, 2023.
Zusammenfassung
The primary goal of this study is to estimate engagement and emotion in a reading task using machine learning techniques and then to utilize the data to design a visualization tool that depicts the differences in engagement and emotion of various readers at regular intervals. A reading experiment with 20 participants and 14 documents was designed which was followed by a questionnaire to rate engagement, arousal, and valence after reading each document on a scale of 1-5. Tobii 4C eye tracker was used along with Empatica E4 wristband to collect data from participants. Different machine learning models were employed to estimate the engagement, arousal, and valence as rated by participants. A 1D-Convolutional Neural Network achieved the highest mean accuracy of 73% for engagement detection, and a Fully Convolution Network network achieved the highest mean accuracy of 66% and 64% for prediction of arousal and valence in a leave-one-participant-out cross-validation. From the evaluation results, a working prototype with an engagement gauge and emoji was developed to visualize the variations in engagement and emotion at timely intervals for each user.