arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2503.00501cs.IRcs.CL
收藏

Qilin:一个包含APP级用户会话的多模态信息检索数据集

Qilin: A Multimodal Information Retrieval Dataset with APP-level User Sessions

Xiaohongshu Inc.(小红书科技有限公司);Tsinghua University(清华大学)

Jia Chen, Qian Dong, Haitao Li, Xiaohui He, Yan Gao, Shaosheng Cao, Yi Wu, Ping Yang, Chen Xu, Yao Hu, Qingyao Ai, Yiqun Liu

AI总结:

本文提出了Qilin,一个从小红书收集的多模态信息检索数据集,包含APP级用户会话、异构结果、上下文信号与真实反馈,支持多模态神经检索及RAG流程训练评估,助力改进搜索与推荐系统。

摘要:

用户生成内容(UGC)社区,尤其是那些以多模态内容为特色的社区,通过将视觉和文本信息整合到结果(或物品)中来提升用户体验。近年来,在具有搜索与推荐(S&R)服务的复杂系统中提升用户体验的挑战,引起了学术界和工业界的极大关注。然而,高质量数据集的缺乏限制了多模态S&R的研究进展。为了满足开发更优S&R服务日益增长的需求,本文提出了一个新颖的多模态信息检索数据集,即Qilin。该数据集收集自小红书,这是一个拥有超过3亿月活跃用户、平均搜索渗透率超过70%的流行社交平台。与现有数据集不同,Qilin提供了全面的用户会话集合,包含异构结果,如图文笔记、视频笔记、商业笔记和直接答案,促进了在各种任务设置下先进的多模态神经检索模型的开发。为了更好地建模用户满意度并支持对异构用户行为的分析,我们还收集了广泛的APP级上下文信号和真实的用户反馈。值得注意的是,Qilin包含了用户偏爱的答案及其针对触发深度问答(DQA)模块的搜索请求所引用的结果。这不仅允许对检索增强生成(RAG)流程进行训练和评估,还允许探索此类模块如何影响用户的搜索行为。通过全面的分析和实验,我们为进一步改进S&R系统提供了有趣的发现和见解。我们希望Qilin在未来能为具有S&R服务的多模态内容平台的发展做出重大贡献。

英文摘要:

User-generated content (UGC) communities, especially those featuring multimodal content, improve user experiences by integrating visual and textual information into results (or items). The challenge of improving user experiences in complex systems with search and recommendation (S\&R) services has drawn significant attention from both academia and industry these years. However, the lack of high-quality datasets has limited the research progress on multimodal S\&R. To address the growing need for developing better S\&R services, we present a novel multimodal information retrieval dataset in this paper, namely Qilin. The dataset is collected from Xiaohongshu, a popular social platform with over 300 million monthly active users and an average search penetration rate of over 70\%. In contrast to existing datasets, \textsf{Qilin} offers a comprehensive collection of user sessions with heterogeneous results like image-text notes, video notes, commercial notes, and direct answers, facilitating the development of advanced multimodal neural retrieval models across diverse task settings. To better model user satisfaction and support the analysis of heterogeneous user behaviors, we also collect extensive APP-level contextual signals and genuine user feedback. Notably, Qilin contains user-favored answers and their referred results for search requests triggering the Deep Query Answering (DQA) module. This allows not only the training \& evaluation of a Retrieval-augmented Generation (RAG) pipeline, but also the exploration of how such a module would affect users' search behavior. Through comprehensive analysis and experiments, we provide interesting findings and insights for further improving S\&R systems. We hope that \textsf{Qilin} will significantly contribute to the advancement of multimodal content platforms with S\&R services in the future.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑