Graph Neural Networks (GNNs) have achieved remarkable success in various applications, but their performance can be sensitive to specific data properties of the graph datasets they operate on. Current literature on understanding the limitations of GNNs has primarily employed a $\textit{model-driven}$ approach that leverage heuristics and domain knowledge from network science or graph theory to model the GNN behaviors, which is time-consuming and highly subjective. In this work, we propose a $\textit{metadata-driven}$ approach to analyze the sensitivity of GNNs to graph data properties, motivated by the increasing availability of graph learning benchmarks. We perform a multivariate sparse regression analysis on the metadata derived from benchmarking GNN performance across diverse datasets, yielding a set of salient data properties. To validate the effectiveness of our data-driven approach, we focus on one identified data property, the degree distribution, and investigate how this property influences GNN performance through theoretical analysis and controlled experiments. Our theoretical findings reveal that datasets with more balanced degree distribution exhibit better linear separability of node representations, thus leading to better GNN performance. We also conduct controlled experiments using synthetic datasets with varying degree distributions, and the results align well with our theoretical findings. Collectively, both the theoretical analysis and controlled experiments verify that the proposed metadata-driven approach is effective in identifying critical data properties for GNNs.
AnaMeta: A Table Understanding Dataset of Field Metadata Knowledge Shared by Multi-dimensional Data Analysis Tasks
Microsoft Research(微软研究院);The University of Hong Kong(香港大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Waterloo(滑铁卢大学);Peking University(北京大学)
Metadata Improves Segmentation Through Multitasking Elicitation
Ukrainian Catholic University(乌克兰天主教大学);Leibniz Institute of Plant Genetics and Crop Plant Research (IPK)(莱布尼茨植物遗传与作物研究所(IPK));University of Tartu(塔尔图大学);Revvity, Inc.(Revvity公司);Wellcome Sanger Institute(威康桑格研究所)
RRCM: Ranking-Driven Retrieval over Collaborative and Meta Memories for LLM Recommendation
The University of Texas at Austin(德克萨斯大学奥斯汀分校);University of Illinois at Chicago(伊利诺伊大学香槟分校);Capital One AI Foundations(Capital One AI基金会);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
When Does Metadata Conditioning (NOT) Work for Language Model Pre-Training? A Study with Context-Free Grammars
The University of Tokyo(东京大学);RIKEN AIP(日本研究机构(Advanced Institute for Proprietary Technology));University of California, Berkeley(加州大学伯克利分校);Preferred Networks, Inc.(Preferred Networks公司)
Large Language Model as Meta-Surrogate for Data-Driven Many-Task Optimization: A Proof-of-Principle Study
School of Computer Science and Engineering, South China University of Technology, Guangzhou, China.(计算机科学与工程学院,华南理工大学,广州,中国);Guangzhou Institute of Technology, Xidian University, China.(广州理工大学,西安电子科技大学,中国);College of Artificial Intelligence, Nankai University, Tianjin, China.(人工智能学院,南开大学,天津,中国);Hanyang University, 15588 Ansan, South Korea.(湖南衡阳大学,韩国)
MetaMP: Seamless Metadata Enrichment and AI Application Framework for Enhanced Membrane Protein Visualization and Analysis
Center for Artificial Intelligence in Public Health Research (ZKI-PH)(公共健康研究人工智能中心);Robert Koch Institute(罗伯特·科赫研究所);University of Marburg(马尔堡大学);Institute of Medical Informatics(医学信息学研究所);University of Münster(穆尔斯特大学);Université Paris Cité(巴黎城市大学);Centre National de la Recherche Scientifique (CNRS)(国家科学研究中心);Department of Mathematics and Computer Science(数学与计算机科学系);Freie Universität Berlin(柏林自由大学)
Explainable Deep Learning for Augmentation of sRNA Expression Profiles
Clausthal University of Technology(克劳斯塔尔工业大学);German Center for Neurodegenerative Diseases(德国神经退行性疾病研究中心);University Medical Center Hamburg-Eppendorf(汉堡-埃彭多夫大学医学中心);Genevention GmbH(Genevention有限责任公司)