Publication
Soft Language Prompts for Language Transfer
Ivan Vykopal; Simon Ostermann; Marián Simko
In: 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (Volume 1: Long Papers). Annual Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL-2025), Association for Computational Linguistics, 2025.
Abstract
Cross-lingual knowledge transfer, especially between high- and low-resource languages, remains challenging in natural language processing (NLP). This study offers insights for improving cross-lingual NLP applications through the combination of parameter-efficient fine-tuning methods. We systematically explore strategies for enhancing cross-lingual transfer through the incorporation of language-specific and task-specific adapters and soft prompts. We present a detailed investigation of various combinations of these methods, exploring their efficiency across 16 languages, focusing on 10 mid- and low-resource languages. We further present to our knowledge the first use of soft prompts for language transfer, a technique we call soft language prompts. Our findings demonstrate that in contrast to claims of previous work, a combination of language and task adapters does not always work best; instead, combining a soft language prompt with a task adapter outperforms most configurations in many cases.
Projects
- DisAI - Improving scientific excellence and creativity in combating disinformation with artificial intelligence and language technologies
- TRAILS - Trustworthy and Inclusive Machines