Foundation Models (FMs) serve as a general class for the development of artificial intelligence systems, offering broad potential for generalization across a spectrum of downstream tasks. Despite extensive research into self-supervised learning as the cornerstone of FMs, several outstanding issues persist in Graph Foundation Models that rely on graph self-supervised learning, namely: 1) Homogenization. The extent of generalization capability on downstream tasks remains unclear. 2) Scalability. It is unknown how effectively these models can scale to large datasets. 3) Efficiency. The training time and memory usage of these models require evaluation. 4) Training Stop Criteria. Determining the optimal stopping strategy for pre-training across multiple tasks to maximize performance on downstream tasks. To address these questions, we have constructed a rigorous benchmark that thoroughly analyzes and studies the generalization and scalability of self-supervised Graph Neural Network (GNN) models. Regarding generalization, we have implemented and compared the performance of various self-supervised GNN models, trained to generate node representations, across tasks such as node classification, link prediction, and node clustering. For scalability, we have compared the performance of various models after training using full-batch and mini-batch strategies. Additionally, we have assessed the training efficiency of these models by conducting experiments to test their GPU memory usage and throughput. Through these experiments, we aim to provide insights to motivate future research. The code for this benchmark is publicly available at https://github.com/NYUSHCS/GraphFM.
Multimodal Clinical Benchmark for Emergency Care (MC-BEC): A Comprehensive Benchmark for Evaluating Foundation Models in Emergency Medicine
Harvard Medical School(哈佛医学院);Stanford University(斯坦福大学);Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University(哈佛大学约翰·A·保尔森工程与应用科学学院)
BioMap Research(BioMap研究部);AI Lab, Research Center for Industries of the Future(人工智能实验室,未来产业研究中心);Westlake University(西湖大学);University of Chinese Academy of Sciences, China(中国科学院大学)
A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT
Michigan State University(密歇根州立大学);Lehigh University(理海大学);Beihang University(北京航空航天大学);Macquarie University(麦考瑞大学);Nanyang Technological University(南洋理工大学);University of California San Diego(加利福尼亚大学圣地亚哥分校);Salesforce AI Research(Salesforce人工智能研究院);Duke University(杜克大学);University of Illinois at Chicago(伊利诺伊大学芝加哥分校)
A Simple Review of EEG Foundation Models: Datasets, Advancements and Future Perspectives
The College of Computer Science, Zhejiang University, Hangzhou, China.(浙江大学计算机科学学院);The MOE Frontiers Science Center for Brain and Brain-Machine Integration, Zhejiang University, Hangzhou, China.(脑科学与脑机接口前沿科学中心);The Nanhu Brain-Computer Interface Institute, Hangzhou, China.(南湖脑机接口研究院);The Qiushi Academy for Advanced Studies, Hangzhou, China.(启石先进研究院)
Low-Rank Adaptation for Foundation Models: A Comprehensive Review
Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Yale University(耶鲁大学);Nanyang Technological University(南洋理工大学);Central University of Finance and Economics(中央财经大学);The Chinese University of Hong Kong(香港中文大学);Xi’an University of Electronic Science and Technology(西安电子科技大学);University of California, Davis(加州大学戴维斯分校);The University of Cambridge(剑桥大学);LOGS AI
Empowering Time Series Analysis with Foundation Models: A Comprehensive Survey
The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));The Hong Kong University of Science and Technology(香港科学与技术大学);Shanghai University of Finance and Economics(上海财经大学)
Unraveling Spatio-Temporal Foundation Models via the Pipeline Lens: A Comprehensive Review
University of Electronic Science and Technology of China(电子科技大学);Aalborg University(奥尔堡大学);Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州))
An Electrocardiogram Multi-task Benchmark with Comprehensive Evaluations and Insightful Findings
Department of Computer Science, Emory University(埃默里大学计算机科学系);Bakar Computational Health Sciences Institute, University of California, San Francisco(加州大学旧金山分校巴卡计算健康科学研究所);Center for Data Science, School of Nursing, Emory University(埃默里大学护理学院数据科学中心);Department of Computer Science, University of Southern California(南加州大学计算机科学系)
A Foundation Chemical Language Model for Comprehensive Fragment-Based Drug Discovery
Advanced Technology Core for Macromolecular X-Ray Crystallography(宏分子X射线衍射先进技术核心);Verna and Marrs McLean Dept. of Biochemistry and Molecular Pharmacology(韦纳和玛里斯麦克莱恩生物化学与分子药理学系);Dept. of Molecular & Cellular Biology(分子与细胞生物学系);Dept. of Molecular Virology & Microbiology(分子病毒学与微生物学系);Baylor College of Medicine(贝勒医学院)
University of Notre Dame(诺丁汉大学);University of Connecticut(康涅狄格大学);University of Virginia(弗吉尼亚大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Cambridge(剑桥大学);Mila - Québec AI Institute(魁北克AI研究院);Université de Montréal(蒙特利尔大学)
A Comprehensive Survey of Foundation Models in Medicine
Herbert Wertheim College of Engineering, University of Florida(佛罗里达大学赫伯特·沃特海姆工程学院);Norman Fixel Institute for Neurological Diseases, University of Florida(佛罗里达大学诺曼·菲克塞尔神经疾病研究所);University of North Carolina at Charlotte(北卡罗来纳大学夏洛特分校);McKnight Brain Institute, University of Florida(佛罗里达大学麦克奈特脑科学研究所)
How to build the best medical image segmentation algorithm using foundation models: a comprehensive empirical study with Segment Anything Model
Department of Electrical and Computer Engineering, Duke University, US(杜克大学电气与计算机工程系);Department of Biostatistics & Bioinformatics, Duke University, US(杜克大学生物统计学与生物信息学系);Department of Computer Science, Duke University, US(杜克大学计算机科学系);Department of Radiology, Duke University, US(杜克大学放射学系)
FISHER: A Foundation Model for Multi-Modal Industrial Signal Comprehensive Representation
Department of Electronic Engineering, Tsinghua University(清华大学电子工程系);Institute for Embodied Intelligence and Robotics, Tsinghua University(清华大学智能感知与机器人研究院);Department of Computer Science and Engineering, Shanghai Jiao Tong University(上海交通大学计算机科学与工程系);Huakong AI Plus Company Limited(华冠AIplus有限公司);Didi International Business Group(滴滴国际商务集团)
BioMatrix: Towards a Comprehensive Biological Foundation Model Spanning the Modality Matrix of Sequences, Structures, and Language
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);OpenDataLab, Shanghai Artificial Intelligence Laboratory(上海人工智能实验室 OpenDataLab);Zhejiang University(浙江大学);Shanghai Innovation Institute(上海创新研究院);East China Normal University(华东师范大学);Zhongguancun Academy(中关村学院);School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院)
Rethinking industrial artificial intelligence: a unified foundation framework
Center for Industrial Artificial Intelligence, Department of Mechanical Engineering, A. James Clark School of Engineering, University of Maryland, College Park, Maryland, United States of America(工业人工智能中心、机械工程系、A. James Clark工程学院、马里兰大学、 College Park分校、马里兰、美国)
Unleashing the Power of Multi-Task Learning: A Comprehensive Survey Spanning Traditional, Deep, and Pretrained Foundation Model Eras
Lehigh University(理海大学);University of Pennsylvania(宾夕法尼亚大学);University of Missouri(密苏里大学);Shandong University(山东大学);University of Hong Kong(香港大学);South China University of Technology(华南理工大学);Cedars-Sinai Medical Center(西达赛奈医疗中心)
A Comprehensive Review of Generative Physical Artificial Intelligence
Qualcomm Research(高通研究院);University of the Cumberlands(坎伯兰大学);BITS-Pilani(皮拉尼比尔拉理工学院);National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学)