SpeakerMem-R1: Speaker-Centered Dual-Track Memory for Multi-Party Dialogue

Problem

This work addresses a gap in the capability of existing models to perform message attribution and relational understanding in multi-party dialogue settings. It specifically targets the challenge of state reconstruction from interleaved dialogue histories. The paper is a preprint and has not undergone peer review.

Method

The authors propose the SpeakerMem-R1 architecture, which utilizes a dual-track memory structure. This architecture is designed to store both speaker-labeled verbatim messages and derived states, organized into person-level and group-level views. The model employs a novel loss function, combining SpeakerLevenshtein and speaker-conditioned GRPO, to optimize performance. The training data consists of three benchmarks: GroupMemBench, SocialMemBench, and EverMemBench. However, the paper does not specify the training compute resources utilized.

Results

The results demonstrate the effectiveness of SpeakerMem-R1 across several benchmarks:

• On GroupMemBench, the model achieves a binary accuracy of 47.9%.
• On SocialMemBench, it reaches a binary accuracy of 69.2%.
• On EverMemBench, the model reports a binary accuracy of 61.9%.
• Notably, on the EverMemBench Leaderboard, SpeakerMem-R1 achieves a score of 62.33%, marking it as the best reported result among state-of-the-art frameworks.
• In a controlled evaluation setting, the model's mean accuracy improved from 57.38% to 68.20% when reinforcement learning (RL) techniques were applied.
• Additionally, the model performs well on the LoCoMo Questions, achieving an accuracy of 70.85% on a dataset of 1,986 questions.

Limitations

The authors do not explicitly state any limitations in their work. However, potential limitations could include the reliance on structured memory construction, which may affect generalizability, and the implications of local deployment scenarios that may not capture the full complexity of multi-party dialogues.

Why it matters

The implications of this research are significant for downstream applications in natural language processing, particularly in enhancing dialogue systems that require nuanced understanding of speaker interactions and message attribution. The dual-track memory approach could pave the way for more sophisticated models capable of handling complex conversational dynamics in real-world applications.