The development of large vision-language models (LVLMs) offers the potential to address challenges faced by traditional multimodal recommendations thanks to their proficient understanding of static images and textual dynamics. However, the application of LVLMs in this field is still limited due to the following complexities: First, LVLMs lack user preference knowledge as they are trained from vast general datasets. Second, LVLMs suffer setbacks in addressing multiple image dynamics in scenarios involving discrete, noisy, and redundant image sequences. To overcome these issues, we propose the novel reasoning scheme named Rec-GPT4V: Visual-Summary Thought (VST) of leveraging large vision-language models for multimodal recommendation. We utilize user history as in-context user preferences to address the first challenge. Next, we prompt LVLMs to generate item image summaries and utilize image comprehension in natural language space combined with item titles to query the user preferences over candidate items. We conduct comprehensive experiments across four datasets with three LVLMs: GPT4-V, LLaVa-7b, and LLaVa-13b. The numerical results indicate the efficacy of VST.
Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation
Rutgers University(罗格斯大学);University of Science and Technology of China(中国科学技术大学);University of Pennsylvania(宾夕法尼亚大学);Bytedance Inc.(字节跳动公司);Georgia Institute of Technology(佐治亚理工学院);University of Pittsburgh(匹兹堡大学);The Hong Kong University of Science and Technology(香港科技大学)
Imagery as Inquiry: Exploring A Multimodal Dataset for Conversational Recommendation
University of California, San Diego(加利福尼亚大学圣迭戈分校);Institute for Clarity in Documentation(文档清晰度研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗夸恩库尔特中心);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室)
LLM4Rec: Large Language Models for Multimodal Generative Recommendation with Causal Debiasing
Department of Software \& Microelectronics Peking University Beijing, China;Department of Software \& Microelectronics Peking University Beijing, China hangli\;Department of Software \& Microelectronics Peking University Beijing, China zehua\;Department of Software \& Microelectronics Peking University Beijing, China xiaofan\;Economic Law School China University of Political Science;Peking University Changsha, China
Leveraging Multimodal Data and Side Users for Diffusion Cross-Domain Recommendation
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications(北京邮电大学计算机学院(国家级试点软件工程学院));The State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全国家重点实验室);Central South University(中南大学);National University of Singapore(新加坡国立大学);Jinan University(暨南大学)
City University of Hong Kong(香港城市大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Michigan State University(密歇根州立大学);Shanghai Jiaotong University(上海交通大学);Jinan University(暨南大学);The University of Queensland(昆士兰大学)
A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarios
Institute of Computational Perception, Johannes Kepler University Linz(约翰开普勒林茨大学计算感知研究所);Human-centered AI Group, AI Lab, Linz Institute of Technology(林茨理工大学人工智能实验室以人为中心的人工智能组)
GUME: Graphs and User Modalities Enhancement for Long-Tail Multimodal Recommendation
Computer Network Information Center, Chinese Academy of Sciences(中国科学院计算机网络信息中心);University of the Chinese Academy of Sciences(中国科学院大学);University of Kansas(堪萨斯大学)
Gated Multimodal Graph Learning for Personalized Recommendation
Miami Herbert Business School, University of Miami, FL, United States(迈阿密赫伯特商学院,迈阿密大学,佛罗里达州,美国);School of Engineering, University of California, California, US(加州大学工程学院,加州,美国);Department of Electrical & Computer Engineering, Rutgers University, Sunnyvale, United States(新泽西理工学院电子与计算机工程系,索菲亚维尔,美国);Department of Electrical and Computer Engineering, Duke University, NC, United States(杜克大学电子与计算机工程系,北卡罗来纳州,美国);School of Engineering, Computer & Mathematical Sciences (ECMS), Auckland University of Technology, Auckland, New Zealand(奥克兰理工大学工程学院,计算机与数学科学(ECMS)系,奥克兰,新西兰);Department Of Computer Science, Universiti Putra Malaysia, Kuala Lumpur, Malaysia(马来西亚国立大学计算机科学系,吉隆坡,马来西亚)
Revisiting Content-Based Music Recommendation: Efficient Feature Aggregation from Large-Scale Music Models
School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室)
A Novel Approach to Balance Convenience and Nutrition in Meals With Long-Term Group Recommendations and Reasoning on Multimodal Recipes and its Implementation in BEACON
Cross-Modal Attention Network with Dual Graph Learning in Multimodal Recommendation
Beijing University of Posts and Telecommunications(北京邮电大学);National University of Singapore(新加坡国立大学);Tianjin University of Technology(天津工业大学);Beihang University(北航);State Key Laboratory of CNS/ATM(国家空管重大科技专项实验室);Aviation Data Communication Corporation(航空数据通信公司)
Multimodality Invariant Learning for Multimedia-Based New Item Recommendation
Hefei University of Technology(合肥工业大学);Hefei Comprehensive National Science Center(合肥综合性国家科学中心);Academy of Cyber(网络空间研究院);Institute of Artificial Intelligence(人工智能研究院)