TeleEgo：在真实场景中评估第一人称AI助手的基准测试
TeleEgo: Benchmarking Egocentric AI Assistants in the Wild
AI总结：
TeleEgo是一个用于评估第一人称AI助手在真实场景中多模态处理能力的基准测试，通过长持续时间流媒体数据和严格评估指标，推动未来具有更强流媒体记忆能力的助手发展。
摘要：
在现实世界中，第一人称AI助手必须处理多模态输入（视频、音频、文本），实时响应，并保留不断演变的长期记忆。然而，现有基准测试通常在孤立状态下评估这些能力，缺乏真实流媒体场景，或仅支持短期任务。我们引入TeleEgo，一个长持续时间、流媒体、多模态的基准测试，用于在真实日常生活场景中评估第一人称AI助手。该数据集包含每个参与者超过14小时的同步第一人称视频、音频和文本，涵盖四个领域：工作与学习、生活方式与日常习惯、社交活动以及外出与文化活动。所有数据都对齐在一个统一的全球时间线上，并包含高质量的视觉叙述和语音转录，通过人工筛选进行整理。TeleEgo定义了12个诊断子任务，涵盖三个核心能力：记忆（回忆过去事件）、理解（解读当前时刻）和跨记忆推理（连接遥远事件）。它包含3291个经过人类验证的QA项目，涵盖多种问题格式（单选、二选一、多选和开放式问题），严格在流媒体环境中评估。我们提出实时准确率（RTA）以共同捕捉正确性和响应速度，同时在紧缩决策窗口下；以及记忆持续时间（MPT）作为衡量连续流中长期保留的前瞻性指标。在本工作中，我们报告了当前模型的RTA结果，并发布TeleEgo，同时发布MPT评估框架，作为未来具有更强流媒体记忆能力的第一人称助手的现实且可扩展的基准测试，使系统研究实时行为和长视界记忆成为可能。
英文摘要：
Egocentric AI assistants in real-world settings must process multi-modal inputs (video, audio, text), respond in real time, and retain evolving long-term memory. However, existing benchmarks typically evaluate these abilities in isolation, lack realistic streaming scenarios, or support only short-term tasks. We introduce \textbf{TeleEgo}, a long-duration, streaming, omni-modal benchmark for evaluating egocentric AI assistants in realistic daily contexts. The dataset features over 14 hours per participant of synchronized egocentric video, audio, and text across four domains: work \& study, lifestyle \& routines, social activities, and outings \& culture. All data is aligned on a unified global timeline and includes high-quality visual narrations and speech transcripts, curated through human refinement.TeleEgo defines 12 diagnostic subtasks across three core capabilities: Memory (recalling past events), Understanding (interpreting the current moment), and Cross-Memory Reasoning (linking distant events). It contains 3,291 human-verified QA items spanning multiple question formats (single-choice, binary, multi-choice, and open-ended), evaluated strictly in a streaming setting. We propose Real-Time Accuracy (RTA) to jointly capture correctness and responsiveness under tight decision windows, and Memory Persistence Time (MPT) as a forward-looking metric for long-term retention in continuous streams. In this work, we report RTA results for current models and release TeleEgo, together with an MPT evaluation framework, as a realistic and extensible benchmark for future egocentric assistants with stronger streaming memory, enabling systematic study of both real-time behavior and long-horizon memory.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
