SAVA-X:通过场景自适应视角对齐和双向跨视角融合进行视角到非视角模仿错误检测
SAVA-X: Ego-to-Exo Imitation Error Detection via Scene-Adaptive View Alignment and Bidirectional Cross View Fusion
AI总结:
本文提出SAVA-X框架,通过场景自适应视角对齐和双向跨视角融合解决视角到非视角模仿错误检测问题,提升了在EgoMe基准上的AUPRC和均值tIoU表现。
摘要:
本文提出SAVA-X框架,通过场景自适应视角对齐和双向跨视角融合解决视角到非视角模仿错误检测问题,提升了在EgoMe基准上的AUPRC和均值tIoU表现。
英文摘要:
Error detection is crucial in industrial training, healthcare, and assembly quality control. Most existing work assumes a single-view setting and cannot handle the practical case where a third-person (exo) demonstration is used to assess a first-person (ego) imitation. We formalize Ego$\rightarrow$Exo Imitation Error Detection: given asynchronous, length-mismatched ego and exo videos, the model must localize procedural steps on the ego timeline and decide whether each is erroneous. This setting introduces cross-view domain shift, temporal misalignment, and heavy redundancy. Under a unified protocol, we adapt strong baselines from dense video captioning and temporal action detection and show that they struggle in this cross-view regime. We then propose SAVA-X, an Align-Fuse-Detect framework with (i) view-conditioned adaptive sampling, (ii) scene-adaptive view embeddings, and (iii) bidirectional cross-attention fusion. On the EgoMe benchmark, SAVA-X consistently improves AUPRC and mean tIoU over all baselines, and ablations confirm the complementary benefits of its components. Code is available at https://github.com/jack1ee/SAVAX.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。