Publikation
From Stateless to Adaptive: Dynamic Personalization for Conversational Language Models
Cicy Agnes; Wolfgang Maaß
In: Women in Machine Learning Workshop (WiML 2024) at NeurIPS. Neural Information Processing Systems (NeurIPS-2024), Thirty-eighth Annual Conference on Neural Information Processing Systems, December 9-15, Vancouver, British Columbia, Canada, Women in Machine Learning Workshop (WiML 2024), 2024.
Zusammenfassung
There are significant challenges in effective communication between humans and conversational AI models due to the lack of consistent personalization and long-term memory retention across interactions. Current language models typically operate without memory, limiting their ability to maintain continuity or deliver personalized conversations over time. Moreover, they rely on static knowledge, lacking mechanisms for tracking user-specific information or evolving contexts. In this work, we propose a method that integrates language models with state tracking mechanisms to enable personalized, context-aware interactions that evolve across multiple sessions.
Our approach includes a memory module that stores and retrieves user-specific data such as preferences, conversational history, and prior interactions. This module enables the model to maintain continuity, adapting dynamically to user-specific needs by tracking both known topics and gaps in knowledge. A state tracking mechanism is employed to organize and recall key user information, such as preferences, interests, and conversational patterns. This allows the model to differentiate between topics that have been discussed, those that are familiar, and areas where knowledge is lacking. This tracks what the model “knows” and “doesn’t know,” enabling it to adapt responses accordingly by filling knowledge gaps over time.
To enhance adaptability and computational efficiency, we integrate Model-Agnostic Meta-Learning (MAML), which allows the system to fine-tune quickly based on minimal user-specific data. During meta-training, the outer loop focuses on domain adaptation, where the model is trained across required domain data. This training equips the model with meta-parameters that generalize well across different contexts. In the inner loop, these meta-parameters are fine-tuned based on individual user interactions, including conversational history and specific preferences. This enables the system to quickly adapt to each user’s needs and data with only a few updates.
As the model interacts with users, it continuously refines its parameters by updating based on user-specific feedback, improving its ability to provide personalized and relevant responses. The meta-parameters are updated through this process, allowing the system to maintain a balance between domain-general knowledge and personalized user adaptation.
The performance of the method can be evaluated through personalized conversational scenarios like chit-chat conversations, student question answering conversations with a chat-bot trained on the specific subjects (example: programming, math) etc and assess its ability to maintain continuity, adapt to new information, and deliver efficient, contextually relevant responses. Our work demonstrates how combining memory mechanisms and state tracking allows conversational models to achieve dynamic, personalized, and contextually aware interactions, overcoming the limitations of stateless systems.