InstrAct：迈向指令视频中的动作中心理解
InstrAct: Towards Action-Centric Understanding in Instructional Videos
AI总结：
本文提出InstrAction框架，通过数据驱动策略和辅助目标提升指令视频中动作识别与建模，建立InstrAct基准测试集，优于现有视频基础模型。
摘要：
理解指令视频需要识别细粒度的动作并建模其时间关系，这仍然是当前视频基础模型（VFMs）的挑战。这一困难源于嘈杂的网络监督和普遍存在的'静态偏差'，即模型依赖于对象而非运动线索。为了解决这一问题，我们提出了InstrAction，一种针对指令视频动作中心表示的预训练框架。我们首先引入了一种数据驱动策略，该策略过滤嘈杂的描述并生成动作中心的困难负样本，在对比学习中分离动作与对象。在视觉特征层面，Action Perceiver从冗余视频编码中提取与运动相关的令牌。除了对比学习之外，我们引入了两个辅助目标：动态时间规整对齐（DTW-Align）用于建模顺序时间结构，以及Masked Action Modeling（MAM）用于加强跨模态关联。最后，我们引入了InstrAct基准测试集来评估动作中心理解，其中我们的方法在语义推理、过程逻辑和细粒度检索任务上始终优于最先进的VFMs。
英文摘要：
Understanding instructional videos requires recognizing fine-grained actions and modeling their temporal relations, which remains challenging for current Video Foundation Models (VFMs). This difficulty stems from noisy web supervision and a pervasive "static bias", where models rely on objects rather than motion cues. To address this, we propose InstrAction, a pretraining framework for instructional videos' action-centric representations. We first introduce a data-driven strategy, which filters noisy captions and generates action-centric hard negatives to disentangle actions from objects during contrastive learning. At the visual feature level, an Action Perceiver extracts motion-relevant tokens from redundant video encodings. Beyond contrastive learning, we introduce two auxiliary objectives: Dynamic Time Warping alignment (DTW-Align) for modeling sequential temporal structure, and Masked Action Modeling (MAM) for strengthening cross-modal grounding. Finally, we introduce the InstrAct Bench to evaluate action-centric understanding, where our method consistently outperforms state-of-the-art VFMs on semantic reasoning, procedural logic, and fine-grained retrieval tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
