ViterbiPlanNet: 通过可微Viterbi注入过程性知识以在教学视频中进行规划
ViterbiPlanNet: Injecting Procedural Knowledge via Differentiable Viterbi for Planning in Instructional Videos
AI总结：
ViterbiPlanNet通过可微Viterbi层整合过程性知识，实现高效的教学视频规划，提升样本效率和鲁棒性。
摘要：
过程性规划旨在预测将初始视觉状态转换为期望目标的一系列动作，这是智能体在复杂环境中运作的基本能力。现有方法通常依赖于大规模模型，通过隐式学习过程性结构，导致样本效率有限且计算成本高。在本工作中，我们引入了ViterbiPlanNet，一个原则性的框架，通过可微Viterbi层（DVL）将过程性知识显式地整合到学习过程中。DVL通过Viterbi解码算法直接嵌入过程性知识图（PKG），用平滑的松弛替代不可微操作，从而实现端到端优化。这种设计使模型能够通过基于图的解码进行学习。在CrossTask、COIN和NIV上的实验表明，ViterbiPlanNet在参数数量比扩散和LLM基于的规划器少一个数量级的情况下实现了最先进的性能。广泛的消融实验表明，性能提升来自于我们的可微结构感知训练，而非事后优化，从而提高了样本效率和对较短未见过的视野的鲁棒性。我们还解决了测试不一致的问题，建立了统一的测试协议，具有一致的划分和评估指标。通过这一新的协议，我们多次运行实验并使用bootstrap方法报告结果以评估统计显著性。
英文摘要：
Procedural planning aims to predict a sequence of actions that transforms an initial visual state into a desired goal, a fundamental ability for intelligent agents operating in complex environments. Existing approaches typically rely on large-scale models that learn procedural structures implicitly, resulting in limited sample-efficiency and high computational cost. In this work we introduce ViterbiPlanNet, a principled framework that explicitly integrates procedural knowledge into the learning process through a Differentiable Viterbi Layer (DVL). The DVL embeds a Procedural Knowledge Graph (PKG) directly with the Viterbi decoding algorithm, replacing non-differentiable operations with smooth relaxations that enable end-to-end optimization. This design allows the model to learn through graph-based decoding. Experiments on CrossTask, COIN, and NIV demonstrate that ViterbiPlanNet achieves state-of-the-art performance with an order of magnitude fewer parameters than diffusion- and LLM-based planners. Extensive ablations show that performance gains arise from our differentiable structure-aware training rather than post-hoc refinement, resulting in improved sample efficiency and robustness to shorter unseen horizons. We also address testing inconsistencies establishing a unified testing protocol with consistent splits and evaluation metrics. With this new protocol, we run experiments multiple times and report results using bootstrapping to assess statistical significance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
