Question-Driven Summarization of Answers to Consumer Health Questions
Lister Hill National Center for Biomedical Communications, U.S. National Library of Medicine, National Institutes of Health（美国国立卫生研究院美国国家医学图书馆利斯特希尔生物医学通信中心）
Max Savery, Asma Ben Abacha, Soumya Gayen, Dina Demner-Fushman
Automatic summarization of natural language is a widely studied area in computer science, one that is broadly applicable to anyone who routinely needs to understand large quantities of information. For example, in the medical domain, recent developments in deep learning approaches to automatic summarization have the potential to make health information more easily accessible to patients and consumers. However, to evaluate the quality of automatically generated summaries of health information, gold-standard, human generated summaries are required. Using answers provided by the National Library of Medicine's consumer health question answering system, we present the MEDIQA Answer Summarization dataset, the first summarization collection containing question-driven summaries of answers to consumer health questions. This dataset can be used to evaluate single or multi-document summaries generated by algorithms using extractive or abstractive approaches. In order to benchmark the dataset, we include results of baseline and state-of-the-art deep learning summarization models, demonstrating that this dataset can be used to effectively evaluate question-driven machine-generated summaries and promote further machine learning research in medical question answering.
Searching for health information online is becoming customary for more and more consumers every day, which makes the need for efficient and reliable question answering systems more pressing. An important contributor to the success rates of these systems is their ability to fully understand the consumers' questions. However, these questions are frequently longer than needed and mention peripheral information that is not useful in finding relevant answers. Question summarization is one of the potential solutions to simplifying long and complex consumer questions before attempting to find an answ…
The growth of online consumer health questions has led to the necessity for reliable and accurate question answering systems. A recent study showed that manual summarization of consumer health questions brings significant improvement in retrieving relevant answers. However, the automatic summarization of long questions is a challenging task due to the lack of training data and the complexity of the related subtasks, such as the question focus and type recognition. In this paper, we introduce a reinforcement learning-based framework for abstractive question summarization. We propose two novel r…
In this paper, we present LiveMedQA, a question answering system that is optimized for consumer health question. On top of the general QA system pipeline, we introduce several new features that aim to exploit domain-specific knowledge and entity structures for better performance. This includes a question type/focus analyzer based on deep text classification model, a tree-based knowledge graph for answer generation and a complementary structure-aware searcher for answer retrieval. LiveMedQA system is evaluated in the TREC 2017 LiveQA medical subtask, where it received an average score of 0.356 …
In this paper, we describe the methodology used and the results obtained by us for completing the tasks given under the shared task on Consumer Health Information Search (CHIS) collocated with the Forum for Information Retrieval Evaluation (FIRE) 2016, ISI Kolkata. The shared task consists of two sub-tasks - (1) task1: given a query and a document/set of documents associated with that query, the task is to classify the sentences in the document as relevant to the query or not and (2) task 2: the relevant sentences need to be further classified as supporting the claim made in the query, or oppo…
A Dataset and Benchmark for Consumer Healthcare Question Summarization
U.S. National Library of Medicine, National Institutes of Health（美国国家医学图书馆，国立卫生研究院）；Department of Computer Science, University of Illinois at Chicago（计算机科学系，伊利诺伊大学芝加哥分校）
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）；Lister Hill National Center for Biomedical Communications（利斯特·希尔国家生物医学通信中心）；National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆）
The quest for seeking health information has swamped the web with consumers' health-related questions. Generally, consumers use overly descriptive and peripheral information to express their medical condition or other healthcare needs, contributing to the challenges of natural language understanding. One way to address this challenge is to summarize the questions and distill the key information of the original question. To address this issue, we introduce a new dataset, CHQ-Summ that contains 1507 domain-expert annotated consumer health questions and corresponding summaries. The dataset is der…
Knowledge of a disease includes information of various aspects of the disease, such as signs and symptoms, diagnosis and treatment. This disease knowledge is critical for many health-related and biomedical tasks, including consumer health question answering, medical language inference and disease name recognition. While pre-trained language models like BERT have shown success in capturing syntactic, semantic, and world knowledge from text, we find they can be further complemented by specific information like knowledge of symptoms, diagnoses, treatments, and other disease aspects. Hence, we int…
Professionals in modern healthcare systems are increasingly burdened by documentation workloads. Documentation of the initial patient anamnesis is particularly relevant, forming the basis of successful further diagnostic measures. However, manually prepared notes are inherently unstructured and often incomplete. In this paper, we investigate the potential of modern NLP techniques to support doctors in this matter. We present a dataset of German patient monologues, and formulate a well-defined information extraction task under the constraints of real-world utility and practicality. In addition,…
Medical Question Summarization with Entity-driven Contrastive Learning
Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)（计算机网络与信息安全部分重点实验室，教育部，山东计算机科学中心（济南国家超算中心），齐鲁工业大学（山东省科学院））；Shandong Provincial Key Laboratory of Computer Networks, Shandong Fundamental Research Center for Computer Science（山东省计算机网络重点实验室，山东省计算机科学基础研究中心）；Australian Artificial Intelligence Institute, University of Technology Sydney（澳大利亚人工智能研究所，悉尼大学技术学院）；Affiliated Hospital of Shandong University of Traditional Chinese Medicine（山东中医药大学附属医院）；School of Computing, Macquaire University（计算机学院，麦考瑞大学）；Data Science Institute, University of Technology Sydney（数据科学研究所，悉尼大学技术学院）
Enhancing Large Language Models with Domain-specific Retrieval Augment Generation: A Case Study on Long-form Consumer Health Question Answering in Ophthalmology
Yale University（耶鲁大学）；National Eye Institute, National Institutes of Health（美国国立卫生研究院国家眼科研究所）；Singapore Eye Research Institute（新加坡眼科研究所）；Singapore National Eye Centre（新加坡国家眼科中心）；National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health（美国国立卫生研究院国家医学图书馆国家生物技术信息中心）；Rice University（莱斯大学）；National University of Singapore（新加坡国立大学）
Harbin Institute of Technology（哈尔滨工业大学）；Hang Seng Management College（恒生管理学院）
Currently, a growing number of health consumers are asking health-related questions online, at any time and from anywhere, which effectively lowers the cost of health care. The most common approach is using online health expert question-answering (HQA) services, as health consumers are more willing to trust answers from professional physicians. However, these answers can be of varying quality depending on circumstance. In addition, as the available HQA services grow, how to predict the answer quality of HQA services via machine learning becomes increasingly important and challenging. In an HQA…
Mental health remains a significant challenge of public health worldwide. With increasing popularity of online platforms, many use the platforms to share their mental health conditions, express their feelings, and seek help from the community and counselors. Some of these platforms, such as Reachout, are dedicated forums where the users register to seek help. Others such as Reddit provide subreddits where the users publicly but anonymously post their mental health distress. Although posts are of varying length, it is beneficial to provide a short, but informative summary for fast processing by…
Extractive summarization is very useful for physicians to better manage and digest Electronic Health Records (EHRs). However, the training of a supervised model requires disease-specific medical background and is thus very expensive. We studied how to utilize the intrinsic correlation between multiple EHRs to generate pseudo-labels and train a supervised model with no external annotation. Experiments on real-patient data validate that our model is effective in summarizing crucial disease-specific information for patients.
Tokyo Metropolitan University（东京都立大学）；RIKEN（理化学研究所）；National Hospital Organization（国立医院机构）；Kitami Institute of Technology（北见工业大学）
Automated summarization of clinical texts can reduce the burden of medical professionals. "Discharge summaries" are one promising application of the summarization, because they can be generated from daily inpatient records. Our preliminary experiment suggests that 20-31% of the descriptions in discharge summaries overlap with the content of the inpatient records. However, it remains unclear how the summaries should be generated from the unstructured source. To decompose the physician's summarization process, this study aimed to identify the optimal granularity in summarization. We first define…
DreamerNLplus: Interpretable Modeling of Mental Health Dynamics from Social Media Timelines using Hybrid Rule-Based and RAG Methods
Leiden Institute of Advanced Computer Science (LIACS), Leiden University（莱顿高级计算机科学研究所（LIACS），莱顿大学）；School of Computing and Communications (SCC), Lancaster University（计算与通信学院（SCC），兰卡斯特大学）；Department of Medical Informatics, Erasmus University Medical Center Rotterdam（医学信息学系，埃因霍温医学中心鲁特万分校）；Biomedical Data Sciences, Leiden University Medical Center（生物医学数据科学，莱顿大学医学中心）
Computer Science and Artificial Intelligence Lab, MIT（麻省理工学院计算机科学与人工智能实验室）；ASAPP, Inc.（ASAPP公司）
We present a method for generating comparative summaries that highlights similarities and contradictions in input documents. The key challenge in creating such summaries is the lack of large parallel training data required for training typical summarization systems. To this end, we introduce a hybrid generation approach inspired by traditional concept-to-text systems. To enable accurate comparison between different sources, the model first learns to extract pertinent relations from input documents. The content planning component uses deterministic operators to aggregate these relations after i…
IIIT-Delhi（德里英迪拉·甘地信息技术学院）；University of Washington（华盛顿大学）
The psychotherapy intervention technique is a multifaceted conversation between a therapist and a patient. Unlike general clinical discussions, psychotherapy's core components (viz. symptoms) are hard to distinguish, thus becoming a complex problem to summarize later. A structured counseling conversation may contain discussions about symptoms, history of mental health issues, or the discovery of the patient's behavior. It may also contain discussion filler words irrelevant to a clinical summary. We refer to these elements of structured psychotherapy as counseling components. In this paper, the…
Domain-specific Guided Summarization for Mental Health Posts
School of Advanced Technology, Xi’an Jiaotong-Liverpool University（西交利物浦大学先进技术学院）；University of Liverpool（利物浦大学）；School of Information Science and Technology, Hangzhou Normal University（杭州师范大学信息科学与技术学院）
The COVID-19 global pandemic has resulted in international efforts to understand, track, and mitigate the disease, yielding a significant corpus of COVID-19 and SARS-CoV-2-related publications across scientific disciplines. As of May 2020, 128,000 coronavirus-related publications have been collected through the COVID-19 Open Research Dataset Challenge. Here we present CO-Search, a retriever-ranker semantic search engine designed to handle complex queries over the COVID-19 literature, potentially aiding overburdened health workers in finding scientific answers during a time of crisis. The retri…
Tencent AI Lab（腾讯人工智能实验室）；University of Central Florida（中佛罗里达大学）
Podcasts have recently shown a rapid rise in popularity. Summarization of podcast transcripts is of practical benefit to both content providers and consumers. It helps consumers to quickly decide whether they will listen to the podcasts and reduces the cognitive load of content providers to write summaries. Nevertheless, podcast summarization faces significant challenges including factual inconsistencies with respect to the inputs. The problem is exacerbated by speech disfluencies and recognition errors in transcripts of spoken language. In this paper, we explore a novel abstractive summarizat…
Overview of the Problem List Summarization (ProbSum) 2023 Shared Task on Summarizing Patients' Active Diagnoses and Problems from Electronic Health Record Progress Notes
University of Wisconsin（威斯康星大学）；Loyola University Chicago（芝加哥洛约拉大学）；Boston Children’s Hospital（波士顿儿童医院）；Harvard Medical School（哈佛医学院）
Automatically generating short summaries from users' online mental health posts could save counselors' reading time and reduce their fatigue so that they can provide timely responses to those seeking help for improving their mental state. Recent Transformers-based summarization models have presented a promising approach to abstractive summarization. They go beyond sentence selection and extractive strategies to deal with more complicated tasks such as novel word generation and sentence paraphrasing. Nonetheless, these models have a prominent shortcoming; their training strategy is not quite ef…
Institute of Automation, CAS（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Zhongke Fanyu Technology Co., Ltd（中科凡语科技有限公司）
Role-oriented dialogue summarization is to generate summaries for different roles in the dialogue, e.g., merchants and consumers. Existing methods handle this task by summarizing each role's content separately and thus are prone to ignore the information from other roles. However, we believe that other roles' content could benefit the quality of summaries, such as the omitted information mentioned by other roles. Therefore, we propose a novel role interaction enhanced method for role-oriented dialogue summarization. It adopts cross attention and decoder self-attention interactions to interacti…
Computer Science and Artificial Intelligence Lab, MIT（麻省理工学院计算机科学与人工智能实验室）；ASAPP, Inc.（ASAPP公司）
We introduce \emph{Nutri-bullets}, a multi-document summarization task for health and nutrition. First, we present two datasets of food and health summaries from multiple scientific studies. Furthermore, we propose a novel \emph{extract-compose} model to solve the problem in the regime of limited parallel data. We explicitly select key spans from several abstracts using a policy network, followed by composing the selected spans to present a summary via a task specific language model. Compared to state-of-the-art methods, our approach leads to more faithful, relevant and diverse summarization -…
Background: Clear language makes communication easier between any two parties. A layman may have difficulty communicating with a professional due to not understanding the specialized terms common to the domain. In healthcare, it is rare to find a layman knowledgeable in medical terminology which can lead to poor understanding of their condition and/or treatment. To bridge this gap, several professional vocabularies and ontologies have been created to map laymen medical terms to professional medical terms and vice versa. Objective: Many of the presented vocabularies are built manually or semi-a…
Open-Access and Collaborative Consumer Health Vocabulary (OAC CHV, or CHV for short), is a collection of medical terms written in plain English. It provides a list of simple, easy, and clear terms that laymen prefer to use rather than an equivalent professional medical term. The National Library of Medicine (NLM) has integrated and mapped the CHV terms to their Unified Medical Language System (UMLS). These CHV terms mapped to 56000 professional concepts on the UMLS. We found that about 48% of these laymen's terms are still jargon and matched with the professional terms on the UMLS. In this pap…
University of Washington（华盛顿大学）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）；University of California, Berkeley（加州大学伯克利分校）；University of Pennsylvania（宾夕法尼亚大学）
When seeking information not covered in patient-friendly documents, like medical pamphlets, healthcare consumers may turn to the research literature. Reading medical papers, however, can be a challenging experience. To improve access to medical papers, we introduce a novel interactive interface-Paper Plain-with four features powered by natural language processing: definitions of unfamiliar terms, in-situ plain language section summaries, a collection of key questions that guide readers to answering passages, and plain language summaries of the answering passages. We evaluate Paper Plain, findi…
Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions
Mashhad University of Medical Sciences（马什哈德医科大学）；Shahid Beheshti University of Medical Sciences（沙希德·贝赫什提医科大学）
Health literacy has emerged as a crucial factor in making appropriate health decisions and ensuring treatment outcomes. However, medical jargon and the complex structure of professional language in this domain make health information especially hard to interpret. Thus, there is an urgent unmet need for automated methods to enhance the accessibility of the biomedical literature to the general population. This problem can be framed as a type of translation problem between the language of healthcare professionals, and that of the general public. In this paper, we introduce the novel task of autom…
Exploring the Efficacy of Large Language Models in Summarizing Mental Health Counseling Sessions: A Benchmark Study
Indian Institute of Technology Delhi（印度理工学院德里分校）；Indraprastha Institute of Information Technology Delhi（英德拉普拉斯塔信息技术学院）；YourDOST；All India Institute of Medical Sciences, Rishikesh（全印医学科学学院里希盖什分校）；All India Institute of Medical Sciences, New Delhi（全印医学科学学院新德里分校）
University of Stavanger（斯塔万格大学）；Links Foundation（林克斯基金会）
In recent years, the trend of deploying digital systems in numerous industries has hiked. The health sector has observed an extensive adoption of digital systems and services that generate significant medical records. Electronic health records contain valuable information for prospective and retrospective analysis that is often not entirely exploited because of the complicated dense information storage. The crude purpose of condensing health records is to select the information that holds most characteristics of the original documents based on a reported disease. These summaries may boost diag…
Automatic medical question summarization can significantly help the system to understand consumer health questions and retrieve correct answers. The Seq2Seq model based on maximum likelihood estimation (MLE) has been applied in this task, which faces two general problems: the model can not capture well question focus and and the traditional MLE strategy lacks the ability to understand sentence-level semantics. To alleviate these problems, we propose a novel question focus-driven contrastive learning framework (QFCL). Specially, we propose an easy and effective approach to generate hard negativ…
Jadavpur University（贾达布尔大学）；Indian Association for the Cultivation of Science（印度科学促进会）；Institute for Advancing Intelligence (IAI), TCG CREST（TCG CREST 智能推进研究所（IAI））
People nowadays use search engines like Google, Yahoo, and Bing to find information on the Internet. Due to explosion in data, it is helpful for users if they are provided relevant summaries of the search results rather than just links to webpages. Text summarization has become a vital approach to help consumers swiftly grasp vast amounts of information.In this paper, different pre-trained models for text summarization are evaluated on different datasets. Specifically, we have used three different pre-trained models, namely, google/pegasus-cnn-dailymail, T5-base, facebook/bart-large-cnn. We ha…
Department of Computer Science, University of Illinois, Chicago, IL, 60607, USA（伊利诺伊大学芝加哥分校计算机科学系）；Indian Institute of Technology, Varanasi, India（印度班加罗尔理工学院）；Department of Computer Science & Engineering, Indraprastha Institute of Information Technology, Delhi, New Delhi, 110020, India（印度德里印度信息技术研究所计算机科学与工程系）
