Graphs play an important role in representing complex relationships in various domains like social networks, knowledge graphs, and molecular discovery. With the advent of deep learning, Graph Neural Networks (GNNs) have emerged as a cornerstone in Graph Machine Learning (Graph ML), facilitating the representation and processing of graphs. Recently, LLMs have demonstrated unprecedented capabilities in language tasks and are widely adopted in a variety of applications such as computer vision and recommender systems. This remarkable success has also attracted interest in applying LLMs to the graph domain. Increasing efforts have been made to explore the potential of LLMs in advancing Graph ML's generalization, transferability, and few-shot learning ability. Meanwhile, graphs, especially knowledge graphs, are rich in reliable factual knowledge, which can be utilized to enhance the reasoning capabilities of LLMs and potentially alleviate their limitations such as hallucinations and the lack of explainability. Given the rapid progress of this research direction, a systematic review summarizing the latest advancements for Graph ML in the era of LLMs is necessary to provide an in-depth understanding to researchers and practitioners. Therefore, in this survey, we first review the recent developments in Graph ML. We then explore how LLMs can be utilized to enhance the quality of graph features, alleviate the reliance on labeled data, and address challenges such as graph Heterophily and out-of-distribution (OOD) generalization. Afterward, we delve into how graphs can enhance LLMs, highlighting their abilities to enhance LLM pre-training and inference. Furthermore, we investigate various applications and discuss the potential future directions in this promising field.
Beyond Superficial Unlearning: Sharpness-Aware Robust Erasure of Hallucinations in Multimodal LLMs
College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics(南京航空航天大学计算机科学与技术学院);Institute for AI, Tsinghua University(清华大学人工智能研究院);Huzhou University(湖州大学);Institute of Dataspace, Hefei Comprehensive National Science Center(合肥综合性国家科学中心数据空间研究院);University of Science and Technology of China(中国科学技术大学)
Huazhong University of Science and Technology(华中科技大学);École Normale Supérieure (ENS) – Université Paris Sciences et Lettres (PSL)(巴黎高等师范学院(ENS)–巴黎文理研究大学(PSL));International School for Advanced Studies (SISSA)(国际高等研究学院(SISSA))
Beyond MedQA: Towards Real-world Clinical Decision Making in the Era of LLMs
Department of Computer Science, Emory University(埃默里大学计算机科学系);Children’s Healthcare of Atlanta(亚特兰大儿童医院);Nell Hodgson Woodruff School of Nursing, Emory University(埃默里大学奈尔·霍德森·伍德鲁夫护理学院)
Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications
Renmin University of China(中国人民大学);The Chinese University of Hong Kong(香港中文大学);Shenzhen University(深圳大学);City University of Hong Kong(香港城市大学);Peking University(北京大学);Massachusetts General Hospital and Harvard Medical School(麻省总医院和哈佛医学院)
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Nanjing University of Science and Technology(南京理工大学);Wuhan AI Research(武汉人工智能研究院)
A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness
The Pennsylvania State University(宾夕法尼亚州立大学);University of Pennsylvania(宾夕法尼亚大学);UTHealth Houston(德克萨斯大学健康科学中心休斯顿分校);Amazon(亚马逊公司);Rensselaer Polytechnic Institute(伦斯勒理工学院)
First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models
Kempner Institute at Harvard University(哈佛大学坎普纳研究所);University of California - Berkeley(加州大学伯克利分校);New York University(纽约大学);Genentech(基因泰克公司);University of Edinburgh(爱丁堡大学)
Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Era
University of Georgia(佐治亚大学);New Jersey Institute of Technology(新泽西理工学院);University of Virginia(弗吉尼亚大学);Wake Forest University(威克森林大学);King Abdullah University of Science and Technology(国王阿卜杜勒阿齐兹大学);Amazon(亚马逊)
Reprogramming Foundational Large Language Models(LLMs) for Enterprise Adoption for Spatio-Temporal Forecasting Applications: Unveiling a New Era in Copilot-Guided Cross-Modal Time Series Representation Learning
Re-Thinking Process Mining in the AI-Based Agents Era
RWTH Aachen University(亚琛工业大学);Fraunhofer FIT(弗劳恩霍夫应用信息技术研究所);Higher Institute of Management of Sousse, University of Sousse(苏塞大学苏塞高等管理学院);Eindhoven University of Technology(埃因霍温理工大学);Process Insights, ECE Group Services(Process Insights公司(ECE集团服务));Microsoft(微软公司)
PERP: Rethinking the Prune-Retrain Paradigm in the Era of LLMs
Department for AI in Society, Science, and Technology, Zuse Institute Berlin, Germany(人工智能与社会、科学与技术部门,柏林Zuse研究所);Institute of Mathematics, Technische Universität Berlin, Germany(数学研究所,柏林技术大学)
MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Tongji University(同济大学)
Mitigating Boundary Ambiguity and Inherent Bias for Text Classification in the Era of Large Language Models
School of Computer Science and Technology, Huazhong University of Science and Technology(华中科技大学计算机科学与技术学院);Joint Laboratory of HUST and Pingan Property & Casualty Research (HPL)(华中科技大学与平安财产保险研究联合实验室);Shanghai AI Laboratory(上海人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Ping An Property & Casualty Insurance Company of China, Ltd.(中国平安财产保险股份有限公司)