面向第一视角视频的个性化问答中的自我定位
Ego-Grounding for Personalized Question-Answering in Egocentric Videos
AI总结：
本文提出MyEgo数据集，用于评估多模态大语言模型在需要自我定位的个性化问答中的能力，发现现有模型在自我记忆和推理方面存在显著不足。
摘要：
我们首次系统分析了多模态大语言模型（MLLMs）在需要自我定位的个性化问答中的表现。为此，我们引入了MyEgo数据集，该数据集旨在评估MLLMs理解、记忆和推理相机佩戴者的能力。MyEgo包含541个长视频和5000个个性化问题，询问关于'我的事物'、'我的活动'和'我的过去'。基准测试显示，各种MLLMs在MyEgo上表现不佳，顶级闭源和开源模型（如GPT-5和Qwen3-VL）的准确率仅为46%和36%，远低于人类表现。令人惊讶的是，显式推理或模型规模扩大并未带来一致提升。模型在相关证据明确提供时表现更好，但随时间推移性能下降，表明在跟踪和记忆'我'和'我的过去'方面存在限制。这些发现突显了自我定位和长程记忆在启用第一视角视频个性化问答中的关键作用。我们希望MyEgo和我们的分析能推动这些领域在第一视角个性化辅助中的进一步进展。数据和代码可在https://github.com/Ryougetsu3606/MyEgo获取。
英文摘要：
We present the first systematic analysis of multimodal large language models (MLLMs) in personalized question-answering requiring ego-grounding - the ability to understand the camera-wearer in egocentric videos. To this end, we introduce MyEgo, the first egocentric VideoQA dataset designed to evaluate MLLMs' ability to understand, remember, and reason about the camera wearer. MyEgo comprises 541 long videos and 5K personalized questions asking about "my things", "my activities", and "my past". Benchmarking reveals that competitive MLLMs across variants, including open-source vs. proprietary, thinking vs. non-thinking, small vs. large scales all struggle on MyEgo. Top closed- and open-source models (e.g., GPT-5 and Qwen3-VL) achieve only~46% and 36% accuracy, trailing human performance by near 40% and 50% respectively. Surprisingly, neither explicit reasoning nor model scaling yield consistent improvements. Models improve when relevant evidence is explicitly provided, but gains drop over time, indicating limitations in tracking and remembering "me" and "my past". These findings collectively highlight the crucial role of ego-grounding and long-range memory in enabling personalized QA in egocentric videos. We hope MyEgo and our analyses catalyze further progress in these areas for egocentric personalized assistance. Data and code are available at https://github.com/Ryougetsu3606/MyEgo
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
