User Simulation for the Evaluation of Bus Information Systems

Jana Götze, Tatjana Scheffler, Roland Roller, Norbert Reithinger

In: Proceedings of 2010 IEEE Workshop on Spoken Language Technology. IEEE Workshop on Spoken Language Technology (SLT-2010) December 12-15 Berkeley CA United States Pages 454-459 IEEE 2010.


In this paper, we describe our contribution to the Spoken Dialog Challenge. We set up a user simulation using the large Let's Go corpus as resource to build our models. Automatic calls were made to all four dialog systems in the SDC, bus information systems that cover the schedule of Pittsburgh, PA. We discuss in detail the architecture and required setup for our system-independent user simulation and report the results and challenges we faced. We also report initial evaluation results.


0000454.pdf (pdf, 153 KB )

German Research Center for Artificial Intelligence
Deutsches Forschungszentrum für Künstliche Intelligenz