Publication
A new metric for the evaluation of dialog act classification.
Stephan Lesch; Thomas Kleinbauer; Jan Alexandersson
In: In Proceedings of the Nineth Workshop on the semantics and pragmatics of dialogue. Workshop on the Semantics and Pragmatics of Dialogue (EDILOG), Nancy, 6/2005.
Abstract
The standard evaluation metrics for dialog act classifiers are based
on the boolean outcome of the exact classification. For
multidimensional tag sets, such as the ICSI-MRDA tag set, this is
stricter than necessary, since the miss- classification might be
partial and this can be good enough for the application in which the
classifier is embedded. We propose a new forgiving metric and show
some preliminary results. Some future work is sketched.