Publication
Assessing Inter-Annotator Agreement for Translation Error Annotation
Arle Richard Lommel; Maja Popovic; Aljoscha Burchardt
In: MTE: Workshop on Automatic and Manual Metrics for Operational Translation Evaluation. International Conference on Language Resources and Evaluation (LREC-14), located at LREC 14, May 26-31, Reykjavik, Iceland, LREC, 2014.
Abstract
One of the key requirements for demonstrating the validity and reliability of an assessment method is that annotators are able to apply it consistently. This paper explores inter-annotator agreement for error classification task and investigates some of the facts that contribute to low IAA.