超越原始信号：作为特权合成数据的未解码生成潜变量
Synesthesia via Direct Latent Augmentation:Bypassing the Decode-Encode Loop for Cross-Modal Distillation
AI总结：
提出直接潜变量增强（DLA）方法，利用未解码的生成潜变量作为特权信息，并通过多层显式模拟联觉（MESSy）将密集知识迁移到纯视觉学生模型，避免了解码-编码循环的低效性。
摘要：
虽然多模态集成显著提升了计算机视觉模型，但部署它们会带来高昂的推理成本，并且需要稀缺且完美配对的数据集。近期方法通过生成式AI合成缺失模态来解决这一数据瓶颈，但它们引入了一个严重的低效问题：解码-编码循环。具体来说，信息丰富的生成潜变量被解码为噪声原始信号，迫使下游分类器浪费容量重新编码它们。为了绕过这一瓶颈，我们提出直接潜变量增强（DLA），直接利用未解码的生成潜变量作为特权信息。此外，为了将这种密集知识迁移到纯视觉学生模型，我们引入多层显式模拟联觉（MESSy）。MESSy 不使用强制表示匹配（这迫使学生扭曲其原生视觉特征以适应复杂的多模态拓扑），而是使用预测目标来安全地内化这些物理先验。实验结果表明，我们的框架显著优于原始数据增强和传统蒸馏。最终，我们的方法产生了高度准确的单模态学生模型，其具有“联觉”潜变量结构，这些结构本质上与它们从未直接观察到的物理属性对齐。
英文摘要：
While multimodal integration significantly improves computer vision models, deploying them incurs prohibitive inference costs and requires scarce, perfectly paired datasets. Recent methods address this data bottleneck by synthesizing missing modalities via generative AI, yet they introduce a severe inefficiency: the Decode-Encode Loop. Specifically, information-rich generative latents are decoded into noisy raw signals, forcing the downstream classifier to waste capacity re-encoding them. To bypass this bottleneck, we propose Direct Latent Augmentation (DLA), utilizing undecoded generative latents directly as privileged information. Furthermore, to transfer this dense knowledge to a purely visual student, we introduce Multilayer Explicit Simulated Synesthesia (MESSy). Instead of enforcing rigid representation matching, which forces the student to distort its native visual features to accommodate complex multimodal topologies, MESSy uses a predictive objective to safely internalize these physical priors. Empirical results demonstrate that our framework significantly outperforms raw data augmentation and traditional distillation. Ultimately, our approach yields highly accurate unimodal students with "synesthetic" latent structures that are inherently aligned with modalities they have never directly observed.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
