AI 中文总结
针对丢包信道协同感知的通用性不足问题，提出SoM-MTM模型，结合掩码图像建模与外部路由MoE机制，提升感知性能与泛化能力，且模型成本可控
AI 中文摘要
为满足下一代移动网络中大规模异构视觉协同感知（CP）的需求，智能高效的传感数据传输是一项关键挑战。在通信网络与智能体人工智能（AI）融合的背景下，现有研究强调利用端到端神经网络简化通信模块，这在协同感知领域展现出良好潜力。然而，这些研究仍局限于特定信道模型、协作模式和感知任务，未能充分利用强大的视觉处理方法提升通用性。为解决该问题，我们提出机器联觉（SoM）驱动的掩码令牌模型（SoM-MTM），作为通用视觉协同感知的即插即用范式。受MAE等掩码图像建模方法启发，该模型具备强大的感知上下文学习能力，可恢复丢包信道上的失真特征，从而提升信息承载效率。在Swin Transformer基础上，SoM-MTM通过外部路由MoE机制嵌入先验掩码信息，在协作过程中最大程度修复并增强环境感知特征。综合实验结果证实，SoM-MTM可在各类任务中持续提升感知性能，尤其对未见过的场景具有强泛化能力，同时保持良好的模型成本与可扩展性。
英文摘要
To support the large-scale and heterogeneous visual cooperative perception (CP) demands in next-generation mobile networks, intelligent and efficient sensory data transmission is a critical challenge. Under the emerging convergence of communication networks and agentic artificial intelligence (AI), existing research emphasizes utilizing end-to-end neural networks to simplify communication modules, which has shown promising potential for CP. However, these studies are still limited to specific channel models, cooperation modes, and perception tasks, failing to fully leverage powerful visual processing approaches to enhance universality. To address this, we propose a Synesthesia of Machines (SoM)-driven Masked Token Model, referred to as SoM-MTM, as a plug-and-play paradigm for generic visual CP. Inspired by masked image modeling methods such as MAE, it possesses great perceptual context learning capabilities to recover distorted features over packet loss channels, thereby improving information carrying efficiency. Building upon Swin Transformer, SoM-MTM further embeds prior masked information through an External Routing MoE mechanism, maximally repairing and enhancing environmental perception features during cooperation. Comprehensive experimental results confirm that SoM-MTM can consistently enhance perception performances on various tasks, especially strong generalization to unseen scenarios, while maintaining favorable model cost and scalability.
