arXiv 2511.124492026-07-31cs.CVcs.AIcs.IRcs.LG

MOON2.0：动态模态平衡多模态表示学习用于电商产品理解

MOON2.0: Dynamic Modality-balanced Multimodal Representation Learning for E-commerce Product Understanding

Alibaba Group（阿里巴巴集团）

Zhanheng Nie, Chenghan Fu, Daoze Zhang, Junxian Wu, Wanxian Guan, Pengjie Wang, Jian Xu, Bo Zheng

备注：: Accepted by the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2026. 11 pages, 7 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

MOON2.0通过动态模态平衡框架解决电商多模态数据中的模态不平衡、信息对齐不足和噪声处理问题，提出MoE、双层对齐和图像-文本共增强策略，提升零样本性能和多模态对齐质量。

摘要：

近期多模态大语言模型（MLLMs）在电商产品理解方面取得了显著进展。然而，它们仍面临三个挑战：（i）由模态混合训练引起的模态不平衡；（ii）在单个产品内视觉和文本信息之间固有的对齐关系未被充分利用；（iii）对电商多模态数据中噪声的处理能力有限。为解决这些问题，我们提出了MOON2.0，一种动态模态平衡的多模态表示学习框架，用于电商产品理解。它包括：（1）一种模态驱动的混合专家（MoE），能够根据输入样本的模态组成动态处理输入样本，从而实现多模态联合学习以缓解模态不平衡；（2）双层对齐方法，以更好地利用单个产品内部的语义对齐属性；（3）基于MLLM的图像-文本共增强策略，通过将文本丰富与视觉扩展相结合，并结合动态样本过滤来提高训练数据质量。我们进一步发布了MBE2.0，一个共增强的多模态表示基准，用于电商表示学习和评估，网址为https://huggingface.co/datasets/ZHNie/MBE2.0。实验表明，MOON2.0在MBE2.0和多个公开数据集上实现了最先进的零样本性能。此外，基于注意力的热图可视化提供了MOON2.0多模态对齐改进的定性证据。

英文摘要：

Recent Multimodal Large Language Models (MLLMs) have significantly advanced e-commerce product understanding. However, they still face three challenges: (i) the modality imbalance induced by modality mixed training; (ii) underutilization of the intrinsic alignment relationships among visual and textual information within a product; and (iii) limited handling of noise in e-commerce multimodal data. To address these, we propose MOON2.0, a dynamic modality-balanced MultimOdal representation learning framework for e-commerce prOduct uNderstanding. It comprises: (1) a Modality-driven Mixture-of-Experts (MoE) that adaptively processes input samples by their modality composition, enabling Multimodal Joint Learning to mitigate the modality imbalance; (2) a Dual-level Alignment method to better leverage semantic alignment properties inside individual products; and (3) an MLLM-based Image-text Co-augmentation strategy that integrates textual enrichment with visual expansion, coupled with Dynamic Sample Filtering to improve training data quality. We further release MBE2.0, a co-augmented Multimodal representation Benchmark for E-commerce representation learning and evaluation at https://huggingface.co/datasets/ZHNie/MBE2.0. Experiments show that MOON2.0 delivers state-of-the-art zero-shot performance on MBE2.0 and multiple public datasets. Furthermore, attention-based heatmap visualization provides qualitative evidence of improved multimodal alignment of MOON2.0.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。