当更多模态反而有害：面向重型车辆发动机诊断的模态丢弃方法

中文摘要 AI 辅助

重型车辆诊断会产生三类互不关联的数据模态：非结构化多语言服务投诉、含超过80%缺失值的高维传感器遥测数据，以及诊断故障码（Diagnostic Trouble Codes，DTC）。我们研究融合这些模态是否能提升某大型卡车制造商专有数据集上的发动机部件分类性能。通过对涵盖三个模型族的多种模型配置进行5折交叉验证，针对五个发动机部件类别（885个样本，为该制造商完整的跨数据库匹配总体），我们发现，朴素融合相比仅使用文本的准确率（65.3%）仅提供小幅提升。然而，训练过程中的模态丢弃（即每个批次随机禁用整个模态）会迫使网络利用较弱的输入，在文本+DTC融合上达到68.8%的准确率（加权F1值：0.67），相比仅使用文本的65.3%（加权F1值：0.64）提升了3.5个百分点，也是包括逻辑回归和梯度提升树在内的所有方法中的最佳结果。按类别分析显示，主导模态因故障类型而异：文本描述症状，DTC编码结构化故障信号，传感器测量物理状态。在进/排气故障上，仅传感器就能达到93%的准确率，而文本仅为80%；在燃油系统故障上，结合模态丢弃的融合方法相比仅使用文本，准确率从15%提升近三倍至38%。据我们所知，这是首次将文本、传感器和故障码三者融合应用于工业车辆诊断的研究。

英文摘要

Heavy-duty vehicle diagnostics generate three disconnected data modalities: unstructured multi- lingual service complaints, high-dimensional sensor telemetry with over 80% missing values, and Diagnostic Trouble Codes (DTCs). We investigate whether fusing these modalities improves engine component classification on a proprietary dataset from a major truck manufacturer. Through 5-fold cross-validation across multiple model configurations spanning three model families on five engine component classes (885 samples, the full cross-database matched population for this manufacturer), we find that naive fusion provides modest gains over text alone (65.3%). However, modality dropout during training, which randomly disables entire modalities per batch, forces the network to exploit weaker inputs and achieves 68.8% accuracy on text+DTC fusion (weighted F1: 0.67), a 3.5-point improvement over text-only (65.3%, weighted F1: 0.64) and the best result across all methods including logistic regression and gradient-boosted trees. Per-class analysis shows that the dominant modality varies by fault type: text describes symptoms, DTCs encode structured fault signals, and sensors measure physical state. On intake/exhaust faults, sensors alone reach 93% where text achieves 80%. On fuel system faults, fusion with modality dropout nearly triples accuracy from 15% to 38% over text alone. To our knowledge, this is the first application of three-way modality fusion combining text, sensors, and fault codes in industrial vehicle diagnostics.

发表机构

• Halmstad University（哈尔姆斯塔德大学）

机构由 AI 辅助整理，请以论文原文为准。