arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.14697cs.IRcs.AI
收藏

当文本作为视觉信号与语义ID在生成推荐中相遇:一项实证研究

When Text-as-Vision Meets Semantic IDs in Generative Recommendation: An Empirical Study

The University of Queensland(昆士兰大学);City University of Hong Kong(香港城市大学);Griffith University(格里菲斯大学)

Shutong Qiao, Wei Yuan, Tong Chen, Xiangyu Zhao, Quoc Viet Hung Nguyen, Hongzhi Yin

AI总结:

本文提出将文本视为视觉信号,通过OCR技术提升生成推荐中语义ID学习的鲁棒性和效率。

摘要:

语义ID学习是生成推荐(GR)模型中的关键接口,将项目映射到基于侧面信息的离散标识符,最常见的是通过预训练的文本编码器。然而,这些文本编码器主要针对结构良好的自然语言进行优化。在现实世界推荐数据中,项目描述通常是符号化的和属性中心的,包含数字、单位和缩写。这些文本编码器会将这些信号拆分为碎片化的token,削弱语义连贯性并扭曲属性间的关系。更糟糕的是,当转向多模态GR时,依赖标准文本编码器会引入额外的障碍:文本和图像嵌入通常表现出不匹配的几何结构,使跨模态融合效果更差且稳定性更低。在本文中,我们重新审视语义ID学习的表示设计,通过将文本视为视觉信号。我们对基于OCR的文本表示进行了系统性的实证研究,通过将项目描述渲染成图像并使用基于视觉的OCR模型进行编码。在四个数据集和两个生成性后端上的实验显示,OCR文本在单模态和多模态设置中均能一致匹配或超越标准文本嵌入用于语义ID学习。此外,我们发现基于OCR的语义ID在极端空间分辨率压缩下仍保持稳健,表明在实际部署中具有强大的鲁棒性和效率。

英文摘要:

Semantic ID learning is a key interface in Generative Recommendation (GR) models, mapping items to discrete identifiers grounded in side information, most commonly via a pretrained text encoder. However, these text encoders are primarily optimized for well-formed natural language. In real-world recommendation data, item descriptions are often symbolic and attribute-centric, containing numerals, units, and abbreviations. These text encoders can break these signals into fragmented tokens, weakening semantic coherence and distorting relationships among attributes. Worse still, when moving to multimodal GR, relying on standard text encoders introduces an additional obstacle: text and image embeddings often exhibit mismatched geometric structures, making cross-modal fusion less effective and less stable. In this paper, we revisit representation design for Semantic ID learning by treating text as a visual signal. We conduct a systematic empirical study of OCR-based text representations, obtained by rendering item descriptions into images and encoding them with vision-based OCR models. Experiments across four datasets and two generative backbones show that OCR-text consistently matches or surpasses standard text embeddings for Semantic ID learning in both unimodal and multimodal settings. Furthermore, we find that OCR-based Semantic IDs remain robust under extreme spatial-resolution compression, indicating strong robustness and efficiency in practical deployments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑