The advent of foundation models (FMs) as an emerging suite of AI techniques has struck a wave of opportunities in computational healthcare. The interactive nature of these models, guided by pre-training data and human instructions, has ignited a data-centric AI paradigm that emphasizes better data characterization, quality, and scale. In healthcare AI, obtaining and processing high-quality clinical data records has been a longstanding challenge, encompassing data quantity, annotation, patient privacy, and ethics. In this survey, we investigate a wide range of data-centric approaches in the FM era (from model pre-training to inference) towards improving the healthcare workflow. We discuss key perspectives in AI security, assessment, and alignment with human values. Finally, we offer a promising outlook on FM-based analytics to enhance patient outcomes and clinical workflows in the evolving landscape of healthcare and medicine. We provide an up-to-date list of healthcare-related foundation models and datasets at https://github.com/Yunkun-Zhang/Data-Centric-FM-Healthcare.
An AI-Guided Data Centric Strategy to Detect and Mitigate Biases in Healthcare Datasets
Icahn School of Medicine at Mount Sinai（西奈山伊坎医学院）；University of Michigan Medical School（密歇根大学医学院）；Beth Israel Deaconess Medical Center / Harvard Medical School（贝斯以色列女执事医疗中心/哈佛医学院）；IBM Research（IBM研究院）
Reinforcement learning (RL) has been extensively researched for enhancing human-environment interactions in various human-centric tasks, including e-learning and healthcare. Since deploying and evaluating policies online are high-stakes in such tasks, off-policy evaluation (OPE) is crucial for inducing effective policies. In human-centric environments, however, OPE is challenging because the underlying state is often unobservable, while only aggregate rewards can be observed (students' test scores or whether a patient is released from the hospital eventually). In this work, we propose a human-…
Explainable artificial intelligence is increasingly used in machine learning (ML) based decision-making systems in healthcare. However, little research has compared the utility of different explanation methods in guiding healthcare experts for patient care. Moreover, it is unclear how useful, understandable, actionable and trustworthy these methods are for healthcare experts, as they often require technical ML knowledge. This paper presents an explanation dashboard that predicts the risk of diabetes onset and explains those predictions with data-centric, feature-importance, and example-based e…
Blockchain-empowered Federated Learning for Healthcare Metaverses: User-centric Incentive Mechanism with Optimal Data Freshness
Guangdong University of Technology（广东工业大学）；Singapore University of Technology and Design（新加坡科技设计大学）；Nanyang Technological University（南洋理工大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
VESSA: Video-based objEct-centric Self-Supervised Adaptation for Visual Foundation Models
Departamento de Ciência da Computação, Universidade Federal de Minas Gerais (UFMG)（巴西联邦大学矿务学院计算机科学系（UFMG））；Recod.ai, Instituto de Computação, Universidade Estadual de Campinas (UNICAMP)（Recod.ai，计算机学院，坎皮纳斯州立大学（UNICAMP））；Google DeepMind（谷歌DeepMind）
An Investigation of Memorization Risk in Healthcare Foundation Models
MIT Broad Institute of MIT and Harvard（MIT Broad研究所（MIT和哈佛））；Vector Institute（向量研究所）；MIT；Chalmers University of Technology（查尔姆斯理工大学）；University of Gothenburg（哥德堡大学）；University of Toronto（多伦多大学）；University Health Network (UHN)（大学健康网络）；Worcester Polytechnic Institute（沃思维尔理工学院）；Computer Science Department（计算机科学系）
Sun Yat-sen University（中山大学）；Hong Kong Baptist University（香港浸会大学）；UC San Diego（加州大学圣地亚哥分校）
As a subfield of machine learning, reinforcement learning (RL) aims at empowering one's capabilities in behavioural decision making by using interaction experience with the world and an evaluative feedback. Unlike traditional supervised learning methods that usually rely on one-shot, exhaustive and supervised reward signals, RL tackles with sequential decision making problems with sampled, evaluative and delayed feedback simultaneously. Such distinctive features make RL technique a suitable candidate for developing powerful solutions in a variety of healthcare domains, where diagnosing decisio…
Predictive models are increasingly used to make various consequential decisions in high-stakes domains such as healthcare, finance, and policy. It becomes critical to ensure that these models make accurate predictions, are robust to shifts in the data, do not rely on spurious features, and do not unduly discriminate against minority groups. To this end, several approaches spanning various areas such as explainability, fairness, and robustness have been proposed in recent literature. Such approaches need to be human-centered as they cater to the understanding of the models to their users. Howev…
Open Foundation Models in Healthcare: Challenges, Paradoxes, and Opportunities with GenAI Driven Personalized Prescription
Qatar University（卡塔尔大学）；Birmingham City University（伯明翰城市大学）；Hamad Bin Khalifa University (HBKU)（哈马德·本·哈利法大学）；University of the West of England (UWE)（西英格兰大学）
Providing front-line health workers in low- and middle- income countries with recommendations and predictions to improve health outcomes can have a tremendous impact on reducing healthcare inequalities, for instance by helping to prevent the thousands of maternal and newborn deaths that occur every day. To that end, we are developing a data-centric machine learning platform that leverages the behavioral logs from a wide range of mobile health applications running in those countries. Here we describe the platform architecture, focusing on the details that help us to maximize the quality and org…
Towards deployment-centric multimodal AI beyond vision and language
Centre for Machine Intelligence, University of Sheffield, Sheffield, UK（机器智能中心，谢菲尔德大学）；School of Computer Science, University of Sheffield, Sheffield, UK（计算机科学学院，谢菲尔德大学）；The Alan Turing Institute, London, UK（艾伦·图灵研究所，伦敦）；Department of Metabolism, Digestion and Reproduction, Imperial College London, London, UK（代谢、消化与生殖部门，伦敦帝国学院）；Department of Applied AI, Simula Research Laboratory, Oslo, Norway（应用人工智能部门，Simula研究实验室，奥斯陆）；Information School, University of Sheffield, Sheffield, UK（信息学院，谢菲尔德大学）；NHS England, Leeds, UK（英国英格兰国家医疗服务体系，利兹）；Institute of Health Informatics, University College London, London, UK（健康信息研究所，伦敦大学学院）；Department of Engineering, King’s College London, London, UK（工程部门，伦敦国王学院）；Department of Computing Science, University of Aberdeen, Aberdeen, UK（计算科学部门，阿伯丁大学）；School of Informatics, University of Edinburgh, Edinburgh, UK（信息学院，爱丁堡大学）；School of Engineering Mathematics and Technology, University of Bristol, Bristol, UK（工程数学与技术学院，布里斯托尔大学）；Department of Informatics, King’s College London, London, UK（信息部门，伦敦国王学院）；Department of Earth Sciences, University of Cambridge, Cambridge, UK（地球科学部门，剑桥大学）；Department of Computer Science, University of Manchester, Manchester, UK（计算机科学部门，曼彻斯特大学）
A Multi-Center Study on the Adaptability of a Shared Foundation Model for Electronic Health Records
Program in Child Health Evaluative Sciences, The Hospital for Sick Children, Toronto, ON, 1 Canada（儿童健康评估科学计划，多伦多医院儿童医学中心，多伦多，加拿大）；Stanford Center for Biomedical Informatics Research, Stanford University, Palo Alto, CA, USA（斯坦福大学生物医学信息学研究中心，斯坦福大学，帕洛阿尔托，美国）；Universidad del Norte, Barranquilla, Columbia（北方大学，巴兰基利亚，哥伦比亚）；Division of Haematology/Oncology, The Hospital for Sick Children, Toronto, ON, Canada（血液肿瘤科，多伦多医院儿童医学中心，多伦多，加拿大）
Towards Data-Centric AI: A Comprehensive Survey of Traditional, Reinforcement, and Generative Approaches for Tabular Data Transformation
University of Kansas（堪萨斯大学）；Southwestern University of Finance and Economics（西南财经大学）；Arizona State University（亚利桑那州立大学）；Portland State University（波特兰州立大学）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；University of Macau（澳门大学）；The Hong Kong University of Science and Technology（香港科技大学）
A Conformation-Centric Generative Foundation Model for Linear Polymer Modeling and Design
Gaoling School of Artificial Intelligence（人工智能学院）；Renmin University of China（中国人民大学）；DP Technology（DP技术）；SINOPEC Research Institute of Petroleum Processing Co., Ltd.（中石油加工研究院）
A Comprehensive Survey of Foundation Models in Medicine
Herbert Wertheim College of Engineering, University of Florida（佛罗里达大学赫伯特·沃特海姆工程学院）；Norman Fixel Institute for Neurological Diseases, University of Florida（佛罗里达大学诺曼·菲克塞尔神经疾病研究所）；University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）；McKnight Brain Institute, University of Florida（佛罗里达大学麦克奈特脑科学研究所）
Geometric-Guided Few-Shot Dental Landmark Detection with Human-Centric Foundation Model
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR, China（电子与计算机工程系，香港科学与技术大学）；Division of Paediatric Dentistry and Orthodontics, Faculty of Dentistry, The University of Hong Kong, Hong Kong SAR, China（牙科学院儿童牙科与正畸学系，香港大学）
Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Models
KTH Royal Institute of Technology（瑞典皇家理工学院）；University of Amsterdam（阿姆斯特丹大学）；Digital Futures（数字未来机构）；Helmholtz AI（亥姆霍兹人工智能研究所）；Technical University of Munich（慕尼黑工业大学）；MPI for Intelligent Systems, Tübingen（蒂宾根马克斯·普朗克智能系统研究所）
University of California, Irvine（加州大学欧文分校）；University of Turku（图尔库大学）
Smart eHealth applications deliver personalized and preventive digital healthcare services to clients through remote sensing, continuous monitoring, and data analytics. Smart eHealth applications sense input data from multiple modalities, transmit the data to edge and/or cloud nodes, and process the data with compute intensive machine learning (ML) algorithms. Run-time variations with continuous stream of noisy input data, unreliable network connection, computational requirements of ML algorithms, and choice of compute placement among sensor-edge-cloud layers affect the efficiency of ML-driven…
A multimodal and temporal foundation model for virtual patient representations at healthcare system scale
Department of Pathology, Mass General Brigham, Harvard Medical School（病理学系，马萨诸塞州总医院与哈佛医学院）；Cancer Program, Broad Institute of Harvard and MIT（癌症计划，哈佛-麻省理工Broad研究所）；Data Science Program, Dana-Farber Cancer Institute（数据科学计划，达纳-法伯癌症研究所）；Health Sciences and Technology, Harvard-MIT（健康科学与技术，哈佛-麻省理工）；Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University（哈佛约翰·A·保罗森工程与应用科学学院，哈佛大学）；Department of Biomedical Informatics, Harvard Medical School（生物医学信息学系，哈佛医学院）；Electrical Engineering and Computer Science, Massachusetts Institute of Technology (MIT)（电气工程与计算机科学，麻省理工学院（MIT））；School of Computer and Communication Sciences, EPFL, Lausanne, Switzerland（计算机与通信科学学院，EPFL，瑞士洛桑）
