LaViC：将大型视觉-语言模型适配到视觉感知的对话式推荐
LaViC: Adapting Large Vision-Language Models to Visually-Aware Conversational Recommendation
AI总结：
提出LaViC框架，通过视觉知识自蒸馏和推荐提示调优，将大型视觉-语言模型适配到对话式推荐，显著优于文本方法和开源基线，并达到专有模型水平。
摘要：
对话式推荐系统通过用户对话来细化需求并提供更个性化的建议。尽管文本信息在许多领域已足够，但时尚或家居装饰等视觉驱动类别可能需要与颜色、风格或设计相关的详细视觉信息。为解决这一挑战，我们提出了LaViC（大型视觉-语言对话推荐框架），一种将紧凑图像表示集成到基于对话的推荐系统中的新方法。LaViC利用大型视觉-语言模型，通过两阶段过程：（1）视觉知识自蒸馏，以自蒸馏方式将产品图像从数百个标记压缩为少量视觉标记，显著降低计算开销；（2）推荐提示调优，使模型能够同时结合对话上下文和蒸馏后的视觉标记，提供统一机制来捕获文本和视觉特征。为支持视觉感知对话推荐系统的严格评估，我们通过将Reddit对话与多个视觉导向类别（如时尚、美容和家居）的Amazon产品列表对齐，构建了一个新数据集。该数据集涵盖了视觉细节至关重要领域中现实用户查询和产品外观。大量实验表明，LaViC显著优于纯文本对话推荐方法和开源视觉-语言基线。此外，与突出的专有基线（如GPT-3.5-turbo、GPT-4o-mini和GPT-4o）相比，LaViC实现了竞争性或更优的准确性，证明了显式使用视觉数据捕获产品属性的必要性，并展示了我们视觉-语言集成的有效性。我们的代码和数据集可在https://github.com/jeon185/LaViC获取。
英文摘要：
Conversational recommender systems engage users in dialogues to refine their needs and provide more personalized suggestions. Although textual information suffices for many domains, visually driven categories such as fashion or home decor potentially require detailed visual information related to color, style, or design. To address this challenge, we propose LaViC (Large Vision-Language Conversational Recommendation Framework), a novel approach that integrates compact image representations into dialogue-based recommendation systems. LaViC leverages a large vision-language model in a two-stage process: (1) visual knowledge self-distillation, which condenses product images from hundreds of tokens into a small set of visual tokens in a self-distillation manner, significantly reducing computational overhead, and (2) recommendation prompt tuning, which enables the model to incorporate both dialogue context and distilled visual tokens, providing a unified mechanism for capturing textual and visual features. To support rigorous evaluation of visually-aware conversational recommendation, we construct a new dataset by aligning Reddit conversations with Amazon product listings across multiple visually oriented categories (e.g., fashion, beauty, and home). This dataset covers realistic user queries and product appearances in domains where visual details are crucial. Extensive experiments demonstrate that LaViC significantly outperforms text-only conversational recommendation methods and open-source vision-language baselines. Moreover, LaViC achieves competitive or superior accuracy compared to prominent proprietary baselines (e.g., GPT-3.5-turbo, GPT-4o-mini, and GPT-4o), demonstrating the necessity of explicitly using visual data for capturing product attributes and showing the effectiveness of our vision-language integration. Our code and dataset are available at https://github.com/jeon185/LaViC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
