arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2509.18436cs.AIcs.CLcs.DB
收藏

Memory-QA:基于多模态记忆回答回忆性问题

Memory-QA: Answering Recall Questions Based on Multimodal Memories

Meta Reality Labs(Meta现实实验室)

Hongda Jiang, Xinyuan Zhang, Siddhant Garg, Rishab Arora, Shiun-Zu Kuo, Jiayang Xu, Ankur Bansal, Christopher Brossman, Yue Liu, Aaron Colak, Ahmed Aly, Anuj Kumar, Xin Luna Dong

AI总结:

本文提出 Memory-QA 真实任务与多模态基准,并通过 Pensieve 流程整合记忆增强、时空感知检索和多记忆问答微调,较现有最优方法最高提升 14% 准确率。

摘要:

我们提出 Memory-QA,这是一项新颖的真实世界任务,旨在根据先前存储的多模态记忆回答有关视觉内容的回忆性问题。该任务带来了独特挑战,包括创建面向任务的记忆、有效利用记忆中的时间与位置信息,以及综合利用多条记忆来回答回忆性问题。为应对这些挑战,我们提出完整流程 Pensieve,集成记忆专用增强、具备时间与位置感知能力的多信号检索,以及多记忆问答微调。我们构建了一个多模态基准以展示该任务中的多种真实挑战,并证明 Pensieve 优于最先进方案,问答准确率最高提升 14%。

英文摘要:

We introduce Memory-QA, a novel real-world task that involves answering recall questions about visual content from previously stored multimodal memories. This task poses unique challenges, including the creation of task-oriented memories, the effective utilization of temporal and location information within memories, and the ability to draw upon multiple memories to answer a recall question. To address these challenges, we propose a comprehensive pipeline, Pensieve, integrating memory-specific augmentation, time- and location-aware multi-signal retrieval, and multi-memory QA fine-tuning. We created a multimodal benchmark to illustrate various real challenges in this task, and show the superior performance of Pensieve over state-of-the-art solutions (up to 14% on QA accuracy).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑