arXiv 2605.089452026-06-23cs.CV

PIDNet: 逐步隐式解耦网络用于多模态动作质量评估

MLCR: Multi-Level Cue Refinement for Long-Term Multimodal Action Quality Assessment

Qiushi Academy for Advanced Studies (QAAS), Zhejiang University（浙江大学启斯特先进研究院）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Software Technology, Zhejiang University（浙江大学软件学院）；State Key Lab of Brain-Machine Intelligence（脑机智能国家重点实验室）；Collaborative Innovation Center for Artificial Intelligence by MOE and Zhejiang Provincial Government (ZJU)（教育部-浙江省人工智能协同创新中心）；Zhejiang Lab（浙江实验室）

Qiqi Li, Pengfei Wang, Hongyu Chen, Nenggan Zheng

备注：: 14 pages, 6 figures, 11 tables

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出PIDNet，通过逐步整合多模态信息与全局质量语义，提升多模态动作质量评估的准确性。采用iMambaWave模块和三阶段融合网络，有效解耦模态特定信息并增强特征表示。

摘要：

动作质量评估（AQA）旨在自动量化视频中人类动作的执行质量，对竞技体育判分具有价值。在多模态AQA中，不同模态的质量证据异质性高，质量线索随时间逐步演变。现有方法常依赖粗略融合或统一时间建模，可能导致模态特定线索模糊、跨模态冗余保留及阶段特定质量证据削弱。为此，我们提出逐步隐式解耦和融合网络（PIDNet），逐步整合模态特定信息、跨模态互补线索和全局质量语义以实现准确评估。具体而言，我们设计了iMambaWave模块，将RGB、光流和音频特征映射到共享潜在空间，并通过Bi-Mamba分支和小波变换分支解耦，分别捕捉长程时间依赖性和局部扰动细节。门控聚合机制适应性融合时域和频域信息。我们进一步构建了三阶段逐步融合网络，使用Group3M块，其中模态互补注意力检索跨模态证据并抑制冗余，多尺度卷积增强特征表示。在Rhythmic Gymnastics和Fis-V数据集上的实验表明，PIDNet在与现有单模态和多模态方法相比，实现了高度竞争的评分相关性和良好的误差控制。消融研究验证了各组件的有效性。此外，iMambaWave在多种主干网络上均能提升视觉表示和时间建模，显示出良好的泛化能力和即插即用能力。

英文摘要：

Long-term multimodal action quality assessment (AQA) evaluates action execution in several-minute audiovisual sequences by mining discriminative quality cues for score prediction. Existing multimodal methods usually model entire sequences with a single temporal encoder and fuse modality features by direct alignment or concatenation, causing key cues to be obscured by global trends, weakened by modal redundancy, and distorted during one-shot score mapping. To address this issue, we reformulate long-term multimodal AQA as a quality cue organization problem and propose MLCR, a multi-level cue refinement framework. MLCR organizes quality evidence at three levels: intra-modal representation, cross-modal interaction, and stage-wise aggregation. Specifically, the intra-modal decoupling encoder (IMDE) preserves modality identity while refining global temporal context and local frequency details. The cross-modal dynamic complementarity-aware retrieval (CMDCR) module retrieves incremental evidence conditioned on the evolving fused state and suppresses redundant responses. The stage-wise multimodal integration (SMI) block progressively accumulates intra-modal and cross-modal cues to refine the fused representation. Experiments on the Rhythmic Gymnastics and Fis-V datasets show that MLCR achieves the best or second-best performance in both Spearman correlation and prediction error, demonstrating its effectiveness and robustness.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。