发表机构
National University of Singapore; University of Chinese Academy of Sciences（新加坡国立大学; 中国科学院大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对多模态数据分块缺失问题，提出MOSAIC框架，通过共享与特定模态表示学习及跨模式校准，在不混淆预测规则的前提下实现跨缺失模式信息借用，理论证明误差分解并验证其在有限目标样本或规则差异场景下的优势。
AI 中文摘要
多模态数据中的分块缺失通常被视为不完全输入问题。我们转而关注预定义观测模态模式的预测，其中观测模态集决定了预测规则可以依赖的信息。对缺失模态进行插补、对未观测模态进行零填充或训练单一池化预测器的方法可以在不同模式间借用信息，但也可能混合模式特定的预测规则。我们提出多模态重叠感知共享特定对齐与模式间校准（MOSAIC），这是一个模式校准框架，用于在缺失模式间借用信息而不混淆其预测规则。MOSAIC学习共享和模态特定的表示，利用与目标模式重叠的可用表示来拟合第一阶段预测器，然后从目标模式数据估计校准差距。我们建立了非渐近界，将误差分解为重叠有效样本量、校准差距和表示学习误差，阐明了跨模式借用何时优于局部拟合，以及何时改进受规则不匹配或表示学习误差控制。模拟实验检验了表示恢复和目标模式校正，在ICU死亡率预测、情绪识别和青光眼分类中的应用表明，当目标模式样本有限或模式特定规则不同时，MOSAIC具有优势。
英文摘要
Blockwise missingness in multimodal data is usually treated as an incomplete-input problem. We instead focus on prediction for a prespecified observed-modality pattern, where the observed modality set determines the information on which the prediction rule can condition. A procedure that imputes missing modalities, zero-fills unobserved modalities, or trains a single pooled predictor may borrow information across patterns, but it can also mix pattern-specific prediction rules. We propose Multimodal Overlap-aware Shared-specific Alignment and Inter-pattern Calibration (MOSAIC), a pattern-calibrated framework for borrowing across missingness patterns without collapsing their prediction rules. MOSAIC learns shared and modality-specific representations, uses the available representations that overlap with the target pattern to fit a first-stage predictor, and then estimates the calibration gap from target-pattern data. We establish non-asymptotic bounds that decompose the error into overlap effective sample size, calibration gap, and representation-learning error, clarifying when cross-pattern borrowing improves over local fitting and when the improvement is controlled by rule mismatch or representation-learning error. Simulations examine representation recovery and target-pattern correction, and applications to ICU mortality prediction, emotion recognition, and glaucoma classification show gains when target-pattern samples are limited or pattern-specific rules differ.
