Publication
Chatbot in the Museum - A Field Study of User Experience and Modality Usage
Stefan Schaffer; Eva Schwaetzer; Aaron Ruß; Oliver Gustke
In: Timo Baumann (Hrsg.). Studientexte zur Sprachkommunikation: Elektronische Sprachsignalverarbeitung 2024. Elektronische Sprachsignalverarbeitung (ESSV-2024), Germany, Pages 13-21, ISBN 978-3-95908-325-6, TUDpress, Dresden, 2024.
Abstract
This paper describes a field study conducted with a museum chatbot at the Städel Museum Frankfurt. The chatbot uses the BERT language model for natural language processing and can be operated via touchscreen as well as via speech input. Prior to the study, hypotheses regarding the user experience of the system were formulated and a system-specific questionnaire was designed, which was used to inquire (among other things) about the perceived quality of the speech output and the frequency of audio guide use in museums. During the interaction with the chatbot, log data was collected and stored in the back-end system. The results show a significant correlation between perceived speech quality and user experience. An exploratory data analysis revealed that participants who used only speech input rated the system as significantly more stimulating than participants who used only touch input. Touch input turned out to be the most efficient input modality in terms of answer correctness and was rated highest regarding pragmatic quality. Interestingly touch input was preferred by younger participants. We discuss our findings and conclude that speech interaction should be seriously considered to create engaging conversational user experiences in museums.