Publication
Retrieval-Augmented Knowledge Integration into Language Models: A Survey
Yuxuan Chen; Daniel Röder; Justus-Jonas Erker; Leonhard Hennig; Philippe Thomas; Sebastian Möller; Roland Roller
In: Sha Li; Manling Li; Michael JQ Zhang; Eunsol Choi; Mor Geva; Peter Hase; Heng Ji (Hrsg.). Proceedings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024). Workshop on Towards Knowledgeable Language Models (KnowLLM-2024), Bangkok, Thailand, Pages 45-63, Association for Computational Linguistics, 2024.
Abstract
This survey analyses how external knowledge can be integrated into language models in the context of retrieval-augmentation.The main goal of this work is to give an overview of: (1) Which external knowledge can be augmented? (2) Given a knowledge source, how to retrieve from it and then integrate the retrieved knowledge? To achieve this, we define and give a mathematical formulation of retrieval-augmented knowledge integration (RAKI). We discuss retrieval and integration techniques separately in detail, for each of the following knowledge formats: knowledge graph, tabular and natural language.