发表机构
Zhejiang University; South China University of Technology; Central China Normal University; Lenovo Group Limited（浙江大学; 华南理工大学; 华中师范大学; 联想集团有限公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对移动和可穿戴设备上个人AI助手回答过去经历查询难的问题，提出LightMem-Ego轻量级流多模态记忆系统，能捕获、对齐并组织视觉和音频流成分层记忆，基于多模态证据生成答案，可部署在多种设备上提供多种支持。
AI 中文摘要
移动和可穿戴设备上的个人人工智能助手通过视觉和音频流持续感知用户的日常生活。然而，回答有关过去经历的查询需要轻量级多模态记忆，能持续积累、组织和检索长期经历，这仍具挑战性。为应对此挑战，我们提出了LightMem-Ego，一种用于日常生活辅助的轻量级流多模态记忆系统。该系统持续捕获以自我为中心的视觉和音频流，在共享时间线上对齐它们，并将其组织成由当前、短期和长期记忆组成的分层记忆。给定用户查询，LightMem-Ego动态地将检索路由到适当的记忆级别，并基于多模态证据生成答案。该演示可部署在智能手机和人工智能眼镜上，支持物体查找、对话回忆、生活总结、日常发现和个性化辅助。代码可在该https网址获取。
英文摘要
Personal AI assistants on mobile and wearable devices continuously perceive users' daily lives through visual and audio streams. However, answering queries about past experiences requires lightweight multimodal memory that can continuously accumulate, organize, and retrieve long-term experiences, which remains challenging. To address this challenge, we present LightMem-Ego, a lightweight streaming multimodal memory system for everyday-life assistance. The system continuously captures egocentric visual and audio streams, aligns them on a shared timeline, and organizes them into a hierarchical memory consisting of current, short-term, and long-term memory. Given a user query, LightMem-Ego dynamically routes retrieval to the appropriate memory level and generates answers grounded in multimodal evidence. The demonstration can be deployed on smartphones and AI glasses, supporting object finding, conversation recall, life summarization, routine discovery, and personalized assistance. Code is available at https://github.com/zjunlp/LightMem-Ego.
CommentsOngoing work
