用于鲁棒多模态序列推荐的序列模态丢弃（Sequential Modality Dropout, SMD）
Sequential Modality Dropout for Robust Multi-Modal Sequential Recommendation
浏览论文内容
中文总结 AI 辅助
本文提出序列模态丢弃（SMD），通过训练时随机擦除模态流提升多模态序列推荐器的鲁棒性，在四个骨干网络和亚马逊数据集上显著提升了缺失模态下的推荐准确率，且几乎不损失完整模态性能。
中文摘要 AI 辅助
多模态序列推荐器假设每个物品都具备所有模态，但实际产品目录中常缺失图像或文本，若在推理时某一模态不可用，基于完整数据训练的模型会大幅降低推荐准确率。本文提出序列模态丢弃（Sequential Modality Dropout, SMD）：训练期间，每个模态流（图像和文本）以概率p独立地对整个用户交互历史进行擦除，使模型学会不依赖单一模态即可预测下一个物品。本文用保留率衡量鲁棒性，即测试时移除某一模态后，模型完整模态准确率（HR@10）的留存比例。在四个骨干网络（MM-SASRec、IISAN、MISSRec和fMRLRec）及四个亚马逊领域的实验中，SMD将文本保留率提升了1.0至3.2倍，且几乎不损失完整模态准确率；在极端的95%单物品缺失率下，SMD的HR@10保留率为61%，而无该方法时仅为22%（提升2.8倍）。可选的跨模态重构损失在简单加性骨干网络上，可在文本严重缺失时将保留率从90%进一步提升至98%。SMD是仅需四行代码、与架构无关的修改，能让多模态序列推荐器在部署时应对实际遇到的缺失模态，提升鲁棒性。
英文摘要
Multi-modal sequential recommenders assume every item carries every modality, but real product catalogs often miss images or text, and a model trained on complete data loses much of its recommendation accuracy when a modality is unavailable at serving time. We propose Sequential Modality Dropout (SMD): during training, each modality stream (image and text) is independently erased with probability p for an entire user interaction history, so the model learns to predict the next item without relying on any single modality. We measure robustness by retention, the fraction of a model's full-modality accuracy (HR@10) that survives when a modality is removed at test time. Across four backbones (MM-SASRec, IISAN, MISSRec, and fMRLRec) on four Amazon domains, SMD raises text retention by 1.0 to 3.2x at essentially no cost to full-modality accuracy; under an extreme 95% per-item missing rate, it retains 61% of HR@10 versus 22% without (a 2.8x improvement). An optional cross-modal reconstruction loss further lifts retention from 90% to 98% on a simple additive backbone under severe text missingness. SMD is a four-line, architecture-agnostic change that makes multi-modal sequential recommenders robust to the missing modalities they actually encounter in deployment.
发表机构
- Stevens Institute of Technology（史蒂文斯理工学院）
机构由 AI 辅助整理，请以论文原文为准。
