EgoPrune：面向具身智能体自运动视频推理的高效令牌剪枝方法
EgoPrune: Efficient Token Pruning for Egomotion Video Reasoning in Embodied Agent
AI总结：
EgoPrune提出一种免训练令牌剪枝方法，通过关键帧选择、透视感知冗余过滤和MMR令牌选择，在保持推理性能的同时显著降低自运动视频推理的计算成本，并验证了其在边缘设备上的实用性。
摘要：
自运动视频是第一人称记录，其中视角因智能体的移动而持续变化。由于它们是具身AI智能体的主要视觉输入，因此提高自运动视频推理的效率对于现实世界部署至关重要。视觉-语言模型的最新进展已实现强大的多模态推理能力，但其计算成本对于冗长、冗余的视频输入仍然过高。现有的令牌剪枝方法通常针对第三人称视频设计，未能利用自运动场景中固有的时空连续性和运动约束。为解决这一问题，我们提出EgoPrune，一种专为自运动视频推理设计的免训练令牌剪枝方法。EgoPrune包含三个组件：一个改编自EmbodiedR的关键帧选择器，用于时间高效采样；透视感知冗余过滤（PARF），利用透视变换对齐视觉令牌并去除冗余令牌；以及一个基于最大边际相关性（MMR）的令牌选择器，联合考虑视觉-文本相关性和帧内多样性。在两个自运动视频基准上的实验表明，EgoPrune在各种剪枝比率下均持续优于先前的免训练方法，同时显著降低FLOPs、内存使用和延迟。此外，我们将EgoPrune部署在一个配备Jetson Orin NX 16GB边缘设备的具身智能体上，展示了其在设备端自运动视频推理中的现实世界效率和适用性。
英文摘要：
Egomotion videos are first-person recordings where the view changes continuously due to the agent's movement. As they serve as the primary visual input for embodied AI agents, making egomotion video reasoning more efficient is therefore essential for real-world deployment. Recent advances in vision-language models have enabled strong multimodal reasoning capabilities, but their computational cost remains prohibitive for long, redundant video inputs. Existing token pruning methods, typically designed for third-person videos, fail to leverage the spatiotemporal continuity and motion constraints inherent in egomotion settings. To address this, we propose EgoPrune, a training-free token pruning method tailored for egomotion video reasoning. EgoPrune comprises three components: a keyframe selector adapted from EmbodiedR for temporally efficient sampling; Perspective-Aware Redundancy Filtering (PARF), which aligns visual tokens using perspective transformations and removes redundant tokens; and a Maximal Marginal Relevance (MMR)-based token selector that jointly considers visual-text relevance and intra-frame diversity. Experiments on two egomotion video benchmarks show that EgoPrune consistently outperforms prior training-free methods across various pruning ratios while significantly reducing FLOPs, memory usage, and latency. Moreover, we deploy EgoPrune on an embodied agent equipped with a Jetson Orin NX 16GB edge device, demonstrating its real-world efficiency and suitability for on-device egomotion video reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
