Skip to main content Skip to main navigation

Project

CORA4NLP

Co(n)textual Reasoning and Adaptation for Natural Language Processing

Co(n)textual Reasoning and Adaptation for Natural Language Processing

  • Duration:
  • Application fields
    Other

Language is implicit - it omits information. Filling this information gap requires contextual inference, background- and commonsense knowledge, and reasoning over situational context. Language also evolves, i.e., it specializes and changes over time. For example, many different languages and domains exist, new domains arise, and both evolve constantly. Thus, language understanding also requires continuous and efficient adaptation to new languages and domains, and transfer to, and between, both. Current language understanding technology, however, focuses on high resource languages and domains, uses little to no context, and assumes static data, task, and target distributions.

The research in Cora4NLP aims to address these challenges. It builds on the expertise and results of the predecessor project DEEPLEE and is carried out jointly between the language technology research departments in Berlin and Saarbrücken. Specifically, our goal is to develop natural language understanding methods that enable:

  • reasoning over broader co- and contexts;
  • efficient adaptation to novel and/or low resource contexts;
  • continuous adaptation to, and generalization over, evolving contexts.

To achieve this, we pursue the following research directions:

  • memory- and language model-augmented few- and zero-shot learning;
  • self- and weakly-supervised pre-training for low-resource domains and long-tail classes;
  • multi-lingual, intra- and inter-document, and dialogue context representations;
  • integration of structured domain knowledge, background- and commonsense knowledge;
  • continual learning for open-domain and supervised tasks multi-hop contextual reasoning.

The resulting methods will be applied in the context of various natural language understanding tasks, such as information extraction, question answering, machine translation, and dialogue.

Sponsors

BMBF - Federal Ministry of Education and Research

01IW20010

BMBF - Federal Ministry of Education and Research

Publications about the project

Alina Leippert; Tatiana Anikina; Bernd Kiefer; Josef van Genabith

In: Yang Cao; Isabel Papadimitriou; Anaelia Ovalle; Marcos Zampieri; Frank Ferraro; Swabha Swayamdipta (Hrsg.). Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (Student Research Workshop). NAACL-HLT Student Research Workshop (NAACL-SRW-2024), located at NAACL, June 18, Mexico City, Mexico, Association for Computational Linguistics, 2024.

To the publication

Tatiana Anikina; Natalia Skachkova; Anna Mokhova

In: Zdeněk ´abokrtský; Maciej Ogrodniczuk (Hrsg.). Proceedings of the CRAC 2023 Shared Task on Multilingual Coreference Resolution. Workshop on Computational Models of Reference, Anaphora and Coreference (CRAC-2023), located at EMNLP 2023, December 6-7, Singapore, Singapore, Pages 19-33, Association for Computational Linguistics, 12/2023.

To the publication

Stalin Varanasi; Muhammad Umer Butt; Günter Neumann

In: Large Language Models for Natural Language Processing. International Conference on Recent Advances in Natural Language Processing (RANLP-2023), located at RANLP, September 4-6, Varna, Bulgaria, Pages 1171-1179, ISBN ISBN 978-954-452-092-2, INCOMA Ltd. Shoumen, BULGARIA, 9/2023.

To the publication