Publication
Occiglot at WMT24: European open-source large language models evaluated on translation
Eleftherios Avramidis; Annika Grützner-Zahn; Manuel Brack; Patrick Schramowski; Pedro Ortiz Suarez; Malte Ostendorff; Fabio Barth; Shushen Manakhimova; Vivien Macketanz; Georg Rehm; Kristian Kersting
In: Philipp Koehn; Barry Haddow; Tom Kocmi; Christof Monz (Hrsg.). Proceedings of the Ninth Conference on Machine Translation. Conference on Machine Translation (WMT-24), located at EMNLP 2024, November 15-16, Miami, Florida, USA, Association for Computational Linguistics, 11/2024.
Abstract
This document describes the submission of the very first version of the Occiglot open-source large language model to the general translation task of the 9th Conference of Machine Translation (WMT24). Occiglot is an open-source, community-based LLM based on Mistral-7B, which went through language-specific continual pre-training and subsequent instruction tuning. We examine the automatic metric scores for translating the WMT24 test set and provide a detailed linguistically-motivated analysis. Despite Occiglot performing worse than many of the other system submissions, we see the submission of this very early version of the model as a motivation to unite community forces and pursue future LLM research on the translation task.