Integrating large language model (LLM) representations into multimodal recommendation has shown promise, yet a fundamental challenge remains largely overlooked: the semantic heterogeneity between generative LM representations and the ID-based collaborative signals that recommendation systems rely on. Naively injecting LM features without alignment degrades recommendation performance rather than improving it. To resolve this, we propose RecGOAT, a dual-granularity semantic alignment framework built on graph neural networks and optimal transport theory. RecGOAT first enriches collaborative semantics through multimodal attentive graphs that capture item-item, user-item, and user-user relationships, initializing user representations via LLM-inferred behavioral preferences. It then aligns LM-derived modality representations with recommendation IDs at two complementary granularities: (1) instance-level alignment via cross-modal contrastive learning (CMCL), which produces discriminative per-sample representations; and (2) distribution-level alignment via optimal adaptive transport (OAT), which minimizes the 1-Wasserstein distance between ID distributions and LLM semantics to produce a unified, consistently aligned feature space. Theoretically, we prove that the unified representation achieves strictly lower target error than any single-modality representation, with the gap bounded by the Wasserstein distance and the InfoNCE loss, providing rigorous guarantees for both alignment consistency and fusion comprehensiveness. Extensive experiments on three public benchmarks demonstrate state-of-the-art performance. Deployment on a large-scale online advertising platform further validates RecGOAT's industrial scalability. Our code is available at https://github.com/6lyc/RecGOAT-LLM4Rec.
Gated Multimodal Graph Learning for Personalized Recommendation
Miami Herbert Business School, University of Miami, FL, United States(迈阿密赫伯特商学院,迈阿密大学,佛罗里达州,美国);School of Engineering, University of California, California, US(加州大学工程学院,加州,美国);Department of Electrical & Computer Engineering, Rutgers University, Sunnyvale, United States(新泽西理工学院电子与计算机工程系,索菲亚维尔,美国);Department of Electrical and Computer Engineering, Duke University, NC, United States(杜克大学电子与计算机工程系,北卡罗来纳州,美国);School of Engineering, Computer & Mathematical Sciences (ECMS), Auckland University of Technology, Auckland, New Zealand(奥克兰理工大学工程学院,计算机与数学科学(ECMS)系,奥克兰,新西兰);Department Of Computer Science, Universiti Putra Malaysia, Kuala Lumpur, Malaysia(马来西亚国立大学计算机科学系,吉隆坡,马来西亚)
Cross-Modal Attention Network with Dual Graph Learning in Multimodal Recommendation
Beijing University of Posts and Telecommunications(北京邮电大学);National University of Singapore(新加坡国立大学);Tianjin University of Technology(天津工业大学);Beihang University(北航);State Key Laboratory of CNS/ATM(国家空管重大科技专项实验室);Aviation Data Communication Corporation(航空数据通信公司)
Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation
Rutgers University(罗格斯大学);University of Science and Technology of China(中国科学技术大学);University of Pennsylvania(宾夕法尼亚大学);Bytedance Inc.(字节跳动公司);Georgia Institute of Technology(佐治亚理工学院);University of Pittsburgh(匹兹堡大学);The Hong Kong University of Science and Technology(香港科技大学)
City University of Hong Kong(香港城市大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Michigan State University(密歇根州立大学);Shanghai Jiaotong University(上海交通大学);Jinan University(暨南大学);The University of Queensland(昆士兰大学)
With the development of information technology, human beings are constantly producing a large amount of information at all times. How to obtain the information that users are interested in from the large amount of information has become an issue of great concern to users and even business managers. In order to solve this problem, from traditional machine learning to deep learning recommendation systems, researchers continue to improve optimization models and explore solutions. Because researchers have optimized more on the recommendation model network structure, they have less research on enri…
GUME: Graphs and User Modalities Enhancement for Long-Tail Multimodal Recommendation
Computer Network Information Center, Chinese Academy of Sciences(中国科学院计算机网络信息中心);University of the Chinese Academy of Sciences(中国科学院大学);University of Kansas(堪萨斯大学)
Leveraging Multimodal Data and Side Users for Diffusion Cross-Domain Recommendation
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications(北京邮电大学计算机学院(国家级试点软件工程学院));The State Key Laboratory of Blockchain and Data Security, Zhejiang University(浙江大学区块链与数据安全国家重点实验室);Central South University(中南大学);National University of Singapore(新加坡国立大学);Jinan University(暨南大学)
Revisiting Content-Based Music Recommendation: Efficient Feature Aggregation from Large-Scale Music Models
School of Artificial Intelligence, Nanjing University(南京大学人工智能学院);National Key Laboratory for Novel Software Technology, Nanjing University(南京大学新型软件技术国家重点实验室)
Carnegie Mellon University(卡内基梅隆大学);University of California, Los Angeles(加州大学洛杉矶分校);Columbia University(哥伦比亚大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Chicago(芝加哥大学);Rice University(莱斯大学)
Accurate news representation is critical for news recommendation. Most of existing news representation methods learn news representations only from news texts while ignore the visual information in news like images. In fact, users may click news not only because of the interest in news titles but also due to the attraction of news images. Thus, images are useful for representing news and predicting user behaviors. In this paper, we propose a multimodal news recommendation method, which can incorporate both textual and visual information of news to learn multimodal news representations. We firs…
Well Begun is Half Done: Training-Free and Model-Agnostic Semantically Guaranteed User Representation Initialization for Multimodal Recommendation
The University of Hong Kong(香港大学);Beijing Institute of Technology(北京理工大学);Carnegie Mellon University(卡内基梅隆大学);Peking University(北京大学);Macao Polytechnic University(澳门理工学院)
As one of the main solutions to the information overload problem, recommender systems are widely used in daily life. In the recent emerging micro-video recommendation scenario, micro-videos contain rich multimedia information, involving text, image, video and other multimodal data, and these rich multimodal information conceals users' deep interest in the items. Most of the current recommendation algorithms based on multimodal data use multimodal information to expand the information on the item side, but ignore the different preferences of users for different modal information, and lack the f…
Multimodal-enhanced Federated Recommendation: A Group-wise Fusion Approach
College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University(吉林大学教育部符号计算与知识工程重点实验室);PolyU Academy for Artificial Intelligence, Hong Kong Polytechnic University(香港理工大学PolyU人工智能学院);Australian Artificial Intelligence Institute, FEIT, University of Technology Sydney(悉尼科技大学FEIT澳大利亚人工智能研究所);College of Computer Science, Inner Mongolia University(内蒙古大学计算机学院)
LLM4Rec: Large Language Models for Multimodal Generative Recommendation with Causal Debiasing
Department of Software \& Microelectronics Peking University Beijing, China;Department of Software \& Microelectronics Peking University Beijing, China hangli\;Department of Software \& Microelectronics Peking University Beijing, China zehua\;Department of Software \& Microelectronics Peking University Beijing, China xiaofan\;Economic Law School China University of Political Science;Peking University Changsha, China
Agents as Knowledge Integrator and Utilizer in Multimodal Recommendation
The University of Hong Kong(香港大学);Beijing Institute of Technology(北京理工大学);Peking University(北京大学);Universiti Malaya(马来亚大学);Macao Polytechnic University(澳门理工学院)
Multimodality Invariant Learning for Multimedia-Based New Item Recommendation
Hefei University of Technology(合肥工业大学);Hefei Comprehensive National Science Center(合肥综合性国家科学中心);Academy of Cyber(网络空间研究院);Institute of Artificial Intelligence(人工智能研究院)