Publikation
SpeechEval: A Domain-Independent User Simulation Platform for Spoken Dialog System Evaluation
Tatjana Scheffler; Roland Roller; Norbert Reithinger
In: Ramon Lopez-Cozar Delgado; Tetsunori Kobayashi (Hrsg.). Proceedings of the Paralinguistic Information and its Integration in Spoken Dialogue Systems Workshop. International Workshop on Spoken Dialogue Systems Technology (IWSDS-11), September 1-3, Granada, Spain, Pages 295-300, ISBN 978-1-4614-1334-9, Springer, 2011.
Zusammenfassung
In this paper, we present the SpeechEval user simulation
platform for spoken dialog system (SDS) evaluation. Our user
simulation realizes a complete end-to-end user test system with spoken
input and output that calls speech-based dialog systems via telephone,
and can be used in the development and evaluation of real-life SDSs.
Our user simulations are realistic in that they can interact on the
speech level, are largely domain and system independent, and can be
used with any given spoken dialog system in a black-box manner, with
relatively little manual adaptation or training data. They can be used
to test SDSs in different stages and to evaluate and compare them, in
addition to costly user tests.