EgoEdit：眼动视频编辑的数据集、实时流模型和基准测试
EgoEdit: Dataset, Real-Time Streaming Model, and Benchmark for Egocentric Video Editing
AI总结：
EgoEdit提出了一套眼动视频编辑的生态系统，包含数据集、实时流模型和基准测试，实现了眼动视频编辑的高稳定性和低延迟，同时在一般编辑任务中保持高性能。
摘要：
我们研究了指导指令的眼动视频编辑，用于交互式AR应用。尽管最近的AI视频编辑器在第三人称镜头上表现良好，但眼动视角带来了独特的挑战-包括快速的自我运动和频繁的手-物体互动-这导致了显著的领域差距。此外，现有的离线编辑流程存在高延迟，限制了实时交互。为了解决这些问题，我们提出了一个完整的生态系统，用于眼动视频编辑。首先，我们构建了EgoEditData，一个精心设计和人工整理的数据集，专门用于眼动编辑场景，具有丰富的手-物体互动，同时明确保留了手部。其次，我们开发了EgoEdit，一个遵循指令的眼动视频编辑器，它支持在单个GPU上进行实时流推断。最后，我们引入了EgoEditBench，一个针对指令忠实性、手部和交互保留以及在自我运动下的时间稳定性进行评估的评估套件。在眼动和一般编辑任务中，EgoEdit产生时间稳定、指令忠实的结果，并具有交互延迟。它在眼动编辑基准测试中实现了显著的改进-现有方法在此处挣扎-同时在一般编辑任务中保持了与最强基线相当的性能。EgoEditData和EgoEditBench将向研究社区公开。请访问我们的网站：https://snap-research.github.io/EgoEdit
英文摘要：
We study instruction-guided editing of egocentric videos for interactive AR applications. While recent AI video editors perform well on third-person footage, egocentric views present unique challenges - including rapid egomotion and frequent hand-object interactions - that create a significant domain gap. Moreover, existing offline editing pipelines suffer from high latency, limiting real-time interaction. To address these issues, we present a complete ecosystem for egocentric video editing. First, we construct EgoEditData, a carefully designed and manually curated dataset specifically designed for egocentric editing scenarios, featuring rich hand-object interactions, while explicitly preserving hands. Second, we develop EgoEdit, an instruction-following egocentric video editor that supports real-time streaming inference on a single GPU. Finally, we introduce EgoEditBench, an evaluation suite targeting instruction faithfulness, hand and interaction preservation, and temporal stability under egomotion. Across both egocentric and general editing tasks, EgoEdit produces temporally stable, instruction-faithful results with interactive latency. It achieves clear gains on egocentric editing benchmarks-where existing methods struggle-while maintaining performance comparable to the strongest baselines on general editing tasks. EgoEditData and EgoEditBench will be made public for the research community. See our website at https://snap-research.github.io/EgoEdit
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
