Objective: The n2c2/UW SDOH Challenge explores the extraction of social determinant of health (SDOH) information from clinical notes. The objectives include the advancement of natural language processing (NLP) information extraction techniques for SDOH and clinical information more broadly. This paper presents the shared task, data, participating teams, performance results, and considerations for future work.
Materials and Methods: The task used the Social History Annotated Corpus (SHAC), which consists of clinical text with detailed event-based annotations for SDOH events such as alcohol, drug, tobacco, employment, and living situation. Each SDOH event is characterized through attributes related to status, extent, and temporality. The task includes three subtasks related to information extraction (Subtask A), generalizability (Subtask B), and learning transfer (Subtask C). In addressing this task, participants utilized a range of techniques, including rules, knowledge bases, n-grams, word embeddings, and pretrained language models (LM).
Results: A total of 15 teams participated, and the top teams utilized pretrained deep learning LM. The top team across all subtasks used a sequence-to-sequence approach achieving 0.901 F1 for Subtask A, 0.774 F1 Subtask B, and 0.889 F1 for Subtask C.
Conclusions: Similar to many NLP tasks and domains, pretrained LM yielded the best performance, including generalizability and learning transfer. An error analysis indicates extraction performance varies by SDOH, with lower performance achieved for conditions, like substance use and homelessness, that increase health risks (risk factors) and higher performance achieved for conditions, like substance abstinence and living with family, that reduce health risks (protective factors).
Extracting Social Determinants of Health from Pediatric Patient Notes Using Large Language Models: Novel Corpus and Methods
University of Washington(华盛顿大学);George Mason University(乔治梅森大学);Dana-Farber Cancer Institute(达纳-法伯癌症研究所);Boston Children's Hospital(波士顿儿童医院);Harvard Medical School(哈佛医学院)
MedDec: A Dataset for Extracting Medical Decisions from Discharge Summaries
Miner School of Computer & Information Sciences, University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校 Miner 计算机与信息科学学院);Institute for Medical Engineering and Science, Massachusetts Institute of Technology(麻省理工学院医学工程与科学研究院)
Integration of Large Language Models and Traditional Deep Learning for Social Determinants of Health Prediction
University of Illinois College of Medicine Peoria(伊利诺伊大学皮奥里亚医学院);Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校西贝尔计算与数据科学学院);Carle Illinois College of Medicine, University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校卡尔·伊利诺伊医学院)
Synth-SBDH: A Synthetic Dataset of Social and Behavioral Determinants of Health for Clinical Text
Manning College of Information and Computer Sciences, University of Massachusetts Amherst(马萨诸塞大学阿姆赫斯特分校信息与计算机科学学院);U.S. Department of Veterans Affairs(美国退伍军人事务部);Department of Medicine, University of Massachusetts Chan Medical School(马萨诸塞大学查恩医学院医学部);Miner School of Computer and Information Sciences, University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校米纳尔计算机与信息科学学院)
Social Determinants of Health Prediction for ICD-9 Code with Reasoning Models
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Illinois Chicago(伊利诺伊大学芝加哥分校);University of Illinois Chicago Peoria(伊利诺伊大学芝加哥分校皮奥里亚分校)
Associations Between Natural Language Processing (NLP) Enriched Social Determinants of Health and Suicide Death among US Veterans
University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);McGill University(麦吉尔大学);University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校);University of Connecticut(康涅狄格大学);Uconn Health(康涅狄格大学健康中心);University of Massachusetts Chan Medical School(马萨诸塞大学陈医学院);Veterans Affairs Bedford Healthcare System(退伍军人事务部贝德福德医疗系统);United States Department of Veterans Affairs(美国退伍军人事务部);University of Texas Health Science Center at Houston(德克萨斯大学休斯顿健康科学中心)
Mining Social Determinants of Health for Heart Failure Patient 30-Day Readmission via Large Language Model
Department of Computer Science, Emory University(计算机科学系,埃默里大学);Nell Hodgson Woodruff School of Nursing, Emory University(Nell Hodgson Woodruff护理学院,埃默里大学)
A Multi-Stage Large Language Model Framework for Extracting Suicide-Related Social Determinants of Health
Cockrell School of Engineering, The University of Texas at Austin(德克萨斯大学奥斯汀分校工程学院);Population Health Sciences, Weill Cornell Medicine(韦尔·柯尔医学中心流行病学与卫生科学系);School of Information, The University of Texas at Austin(德克萨斯大学奥斯汀分校信息学院);Department of Biomedical Informatics, Columbia University(哥伦比亚大学生物医学信息学系)
SDoH-GPT: Using Large Language Models to Extract Social Determinants of Health (SDoH)
University of Texas at Austin(德克萨斯大学奥斯汀分校);Pontifical Catholic University of Rio Grande do Sul(南里奥格兰德天主教大学);University of Pittsburgh(匹兹堡大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);University of Pennsylvania(宾夕法尼亚大学);University of Texas Southwestern(德克萨斯大学西南医学中心);University of Virginia(弗吉尼亚大学);Temple University(天普大学);Weill Cornell Medicine(威尔康奈尔医学院)
Multi-Task Learning for Extracting Menstrual Characteristics from Clinical Notes
Blavatnik Family Women’s Health Research Institute, Icahn School of Medicine at Mount Sinai(西奈山伊坎医学院布拉瓦特尼克家族女性健康研究所);Hasso Plattner Institute for Digital Health at Mount Sinai, Icahn School of Medicine at Mount Sinai(西奈山伊坎医学院西奈山哈索·普拉特纳数字健康研究所);Department for Artificial Intelligence and Human Health, Icahn School of Medicine at Mount Sinai(西奈山伊坎医学院人工智能与人类健康系);Department of Obstetrics, Gynecology, and Reproductive Science, Icahn School of Medicine at Mount Sinai(西奈山伊坎医学院妇产科与生殖科学系)
QuaLLM-Health: An Adaptation of an LLM-Based Framework for Quantitative Data Extraction from Online Health Discussions
The University of Texas MD Anderson Cancer Center(德克萨斯大学MD安德森癌症中心);Texas Tech Health Science Center El Paso(德克萨斯理工大学埃尔帕索健康科学中心);University of Texas at Austin(德克萨斯大学奥斯汀分校)
Extracting Social Support and Social Isolation Information from Clinical Psychiatry Notes: Comparing a Rule-based NLP System and a Large Language Model
Weill Cornell Medicine(威尔康奈尔医学院);Icahn School of Medicine at Mount Sinai(西奈山伊坎医学院);Mayo Clinic(梅奥诊所);Columbia University Vagelos College of Physicians and Surgeons(哥伦比亚大学瓦格洛斯内科医生与外科医生学院);New York State Psychiatric Institute(纽约州精神病学研究所)
A Dataset and Resources for Identifying Patient Health Literacy Information from Clinical Notes
National Library of Medicine(国家医学图书馆);Massachusetts Institute of Technology(麻省理工学院);Yonsei University College of Medicine(延世大学医学院);Beth Israel Deaconess Medical Center(贝斯以色列医疗中心);NYC Health + Hospitals - Lincoln(纽约市卫生局-林肯医院)
Towards conversational assistants for health applications: using ChatGPT to generate conversations about heart failure
Department of Computer Science(计算机科学系);Department of Biomedical and Health Information Sciences(生物医学与健康信息科学系);Department of Medicine(医学系);Department of Communications(通讯系)
Extracting information from free text through unsupervised graph-based clustering: an application to patient incident records
Imperial College London(帝国理工学院);Department of Mathematics, Imperial College London, UK(帝国理工学院数学系);Department of Computing, Imperial College London, UK(帝国理工学院计算机系);Department of Surgery & Cancer, Imperial College London, UK(帝国理工学院外科与癌症系);Department of Chemistry, Imperial College London, UK(帝国理工学院化学系)
MasonNLP+ at SemEval-2023 Task 8: Extracting Medical Questions, Experiences and Claims from Social Media using Knowledge-Augmented Pre-trained Language Models
Automated Identification of Eviction Status from Electronic Health Record Notes
Manning College of Information and Computer Sciences, University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校曼宁信息与计算机科学学院);National Center on Homelessness among Veterans, U.S. Department of Veterans Affairs Homeless Programs Office(美国退伍军人事务部无家可归者项目办公室退伍军人无家可归问题国家中心);School of Public Health, University of Texas Health Science Center at Houston(德克萨斯大学休斯顿健康科学中心公共卫生学院);Yale University School of Medicine(耶鲁大学医学院);University of Massachusetts Medical School(马萨诸塞大学医学院);Center for Biomedical and Health Research in Data Sciences, Miner School of Computer and Information Sciences, University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校迈纳计算机与信息科学学院生物医学与健康数据科学研究中心);Center for Healthcare Organization and Implementation Research, VA Bedford Health Care(退伍军人事务部贝德福德医疗保健系统医疗保健组织与实施研究中心)
Center for Computational Engineering and Networking(计算工程与网络中心);Amrita School of Engineering Coimbatore(阿姆里塔工程学院哥印拜陀校区);Amrita Vishwa Vidyapeetham(阿姆里塔大学);Amrita University(阿姆里塔大学)