Publication
Linguistic evaluation of German-English Machine Translation using a Test Suite
Eleftherios Avramidis; Vivien Macketanz; Ursula Strohriegel; Hans Uszkoreit
In: Proceedings of the Fourth Conference on Machine Translation. Conference on Machine Translation (WMT-2019), August 1-2, Florence, Italy, Association for Computational Linguistics, 8/2019.
Abstract
We present the results of the application of a grammatical test suite for German$\rightarrow$English MT on the systems submitted at WMT19, with a detailed analysis for 107 phenomena organized in 14 categories.
The systems still translate wrong one out of four test items in average. Low performance is indicated for idioms, modals, pseudo-clefts, multi-word expressions and verb valency. When compared to last year, there has been a improvement of function words, non-verbal agreement and punctuation. More detailed conclusions about particular systems and phenomena are also presented.
Projects
- DEEPLEE - Tiefes Lernen für End-to-End-Anwendungen in der Sprachtechnologie
- BBDCII - Berliner Big Data Center