对齐与适应：在任意视图组合下的人类活动识别多模态多视角学习
Align and Adapt: Multimodal Multiview Human Activity Recognition under Arbitrary View Combinations
AI总结：
AliAd通过结合多视角对比学习和专家混合模块，实现任意视图组合下的灵活多模态人类活动识别，提升模型适应性和计算效率。
摘要：
多模态多视角学习旨在整合来自不同来源的信息以提高任务性能。现有方法往往难以处理灵活的视图配置，包括任意视图组合、视图数量和异构模态。聚焦于人类活动识别的背景，我们提出AliAd模型，该模型结合了多视角对比学习和专家混合模块，以支持训练和推理过程中任意视图的可用性。与其试图重建缺失的视图，我们采用调整后的中心对比损失进行自监督表示学习和视图对齐，减轻缺失视图对多视图融合的影响。这种损失公式允许整合视图权重以考虑视图质量。此外，它将计算复杂度从O(V²)降低到O(V)，其中V是视图的数量。为了解决对比学习无法捕捉的残余差异，我们采用专家混合模块，并采用专门的负载平衡策略，负责适应任意视图组合。我们强调模型中组件之间的几何关系以及它们在潜在空间中的良好结合。AliAd在四个数据集上得到验证，涵盖惯性与人体姿态模态，视图数量从三到九不等，展示了其性能和灵活性。
英文摘要：
Multimodal multiview learning seeks to integrate information from diverse sources to enhance task performance. Existing approaches often struggle with flexible view configurations, including arbitrary view combinations, numbers of views, and heterogeneous modalities. Focusing on the context of human activity recognition, we propose AliAd, a model that combines multiview contrastive learning with a mixture-of-experts module to support arbitrary view availability during both training and inference. Instead of trying to reconstruct missing views, an adjusted center contrastive loss is used for self-supervised representation learning and view alignment, mitigating the impact of missing views on multiview fusion. This loss formulation allows for the integration of view weights to account for view quality. Additionally, it reduces computational complexity from $O(V^2)$ to $O(V)$, where $V$ is the number of views. To address residual discrepancies not captured by contrastive learning, we employ a mixture-of-experts module with a specialized load balancing strategy, tasked with adapting to arbitrary view combinations. We highlight the geometric relationship among components in our model and how they combine well in the latent space. AliAd is validated on four datasets encompassing inertial and human pose modalities, with the number of views ranging from three to nine, demonstrating its performance and flexibility.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
