Publication
75 Languages, 1 Model: Parsing Universal Dependencies Universally
Daniel Kondratyuk; Milan Straka
In: Proceedings of the Conference on Empirical Methods in Natural Language Processing 2019. Conference on Empirical Methods in Natural Language Processing (EMNLP-2019), November 3-7, Hong Kong, China, Pages 2779-2795, ISBN 978-1-950737-90-1, Association for Computational Linguistics, Stroudsburg, PA, USA, 11/2019.
Abstract
We present UDify, a multilingual multi-task model capable of accurately predicting universal part-of-speech, morphological features, lemmas, and dependency trees simultaneously for all 124 Universal Dependencies treebanks across 75 languages. By leveraging multilingual BERT self-attention model pretrained on 104 languages, we found that fine-tuning it on all datasets concatenated together with simple softmax classifiers for each UD task can meet or exceed state-of-the-art UPOS, UFeats, Lemmas, (and especially)
UAS, and LAS scores, without requiring any recurrent or language-specific components. We evaluateUDify for multilingual learning, showing that low-resource languages benefit the most from cross-linguistic annotations. We also evaluate for zero-shot learning, with results suggesting that multilingual training provides strong UD predictions even for languages that neither UDify nor BERT have ever been trained on. Code for UDify is available at https://github.com/hyperparticle/udify.