A Corpus for Detecting High-Context Medical Conditions in Intensive Care Patient Notes Focusing on Frequently Readmitted Patients
Massachusetts Institute of Technology（麻省理工学院）；University of Massachusetts（马萨诸塞大学）；Harvard T.H. Chan School of Public Health（哈佛大学陈曾熙公共卫生学院）；Massachusetts General Hospital（麻省总医院）；Tufts University School of Medicine（塔夫茨大学医学院）；Beth Israel Deaconess Medical Center（贝斯以色列女执事医疗中心）；Washington University School of Medicine（华盛顿大学医学院）；Philips Research North America（飞利浦北美研究院）；Harvard University（哈佛大学）；Adobe Research（Adobe研究院）
Edward T. Moseley, Joy T. Wu, Jonathan Welt, John Foote, Patrick D. Tyler, David W. Grant, Eric T. Carlson, Sebastian Gehrmann, Franck Dernoncourt, Leo Anthony Celi
A crucial step within secondary analysis of electronic health records (EHRs) is to identify the patient cohort under investigation. While EHRs contain medical billing codes that aim to represent the conditions and treatments patients may have, much of the information is only present in the patient notes. Therefore, it is critical to develop robust algorithms to infer patients' conditions and treatments from their written notes. In this paper, we introduce a dataset for patient phenotyping, a task that is defined as the identification of whether a patient has a given medical condition (also referred to as clinical indication or phenotype) based on their patient note. Nursing Progress Notes and Discharge Summaries from the Intensive Care Unit of a large tertiary care hospital were manually annotated for the presence of several high-context phenotypes relevant to treatment and risk of re-hospitalization. This dataset contains 1102 Discharge Summaries and 1000 Nursing Progress Notes. Each Discharge Summary and Progress Note has been annotated by at least two expert human annotators (one clinical researcher and one resident physician). Annotated phenotypes include treatment non-adherence, chronic pain, advanced/metastatic cancer, as well as 10 other phenotypes. This dataset can be utilized for academic and industrial research in medicine and computer science, particularly within the field of medical natural language processing.
A Large and Balanced Corpus for Fine-grained Arabic Readability Assessment
Computational Approaches to Modeling Language Lab, New York University Abu Dhabi（新纽约大学阿布扎赫实验室）；Zai Arabic Language Research Centre, Zayed University（扎伊阿拉伯语言研究中心）
Italian Institute of Technology（意大利理工学院）；University of Ferrara（费拉拉大学）；University of Genoa（热那亚大学）；Azienda Ospedaliera Universitaria Sant'Anna（圣安娜大学附属医院）；PerVoice（PerVoice公司）
This paper introduces a new dysarthric speech command dataset in Italian, called EasyCall corpus. The dataset consists of 21386 audio recordings from 24 healthy and 31 dysarthric speakers, whose individual degree of speech impairment was assessed by neurologists through the Therapy Outcome Measure. The corpus aims at providing a resource for the development of ASR-based assistive technologies for patients with dysarthria. In particular, it may be exploited to develop a voice-controlled contact application for commercial smartphones, aiming at improving dysarthric patients' ability to communica…
Virginia Tech（弗吉尼亚理工大学）；IBM Research Almaden（IBM阿尔马登研究院）；eBay Inc（eBay公司）
Recent advances in text representation have shown that training on large amounts of text is crucial for natural language understanding. However, models trained without predefined notions of topical interest typically require careful fine-tuning when transferred to specialized domains. When a sufficient amount of within-domain text may not be available, expanding a seed corpus of relevant documents from large-scale web data poses several challenges. First, corpus expansion requires scoring and ranking each document in the collection, an operation that can quickly become computationally expensiv…
When speaking or writing, people omit information that seems clear and evident, such that only part of the message is expressed in words. Especially in argumentative texts it is very common that (important) parts of the argument are implied and omitted. We hypothesize that for argument analysis it will be beneficial to reconstruct this implied information. As a starting point for filling such knowledge gaps, we build a corpus consisting of high-quality human annotations of missing and implied information in argumentative texts. To learn more about the characteristics of both the argumentative …
University of Southern California（南加利福尼亚大学）；Information Sciences Institute, University of Southern California（南加利福尼亚大学信息科学研究所）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Human activities can be seen as sequences of events, which are crucial to understanding societies. Disproportional event distribution for different demographic groups can manifest and amplify social stereotypes, and potentially jeopardize the ability of members in some groups to pursue certain goals. In this paper, we present the first event-centric study of gender biases in a Wikipedia corpus. To facilitate the study, we curate a corpus of career and personal life descriptions with demographic information consisting of 7,854 fragments from 10,412 celebrities. Then we detect events with a stat…
The fifth Dialog State Tracking Challenge (DSTC5) introduces a new cross-language dialog state tracking scenario, where the participants are asked to build their trackers based on the English training corpus, while evaluating them with the unlabeled Chinese corpus. Although the computer-generated translations for both English and Chinese corpus are provided in the dataset, these translations contain errors and careless use of them can easily hurt the performance of the built trackers. To address this problem, we propose a multichannel Convolutional Neural Networks (CNN) architecture, in which …
This paper explores contexts associated with errors in transcrip-tion of spontaneous speech, shedding light on human perceptionof disfluencies and other conversational speech phenomena. Anew version of the Switchboard corpus is provided with disfluency annotations for careful speech transcripts, together with results showing the impact of transcription errors on evaluation of automatic disfluency detection.
JobHop v2: A Large-Scale Career Trajectory Dataset from Unstructured Resumes
AIDA-IDLab, Department of Electronics and Information Systems, Ghent University, Ghent, Belgium（AIDA-ID实验室，电子与信息系统系，根特大学，根特，比利时）；AIDA-IDLab, Department of Electronics（AIDA-ID实验室，电子系）；Information Systems, Ghent University, Ghent, Belgium（信息系统，根特大学，根特，比利时）
Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions
Mashhad University of Medical Sciences（马什哈德医科大学）；Shahid Beheshti University of Medical Sciences（沙希德·贝赫什提医科大学）
MentalQA: An Annotated Arabic Corpus for Questions and Answers of Mental Healthcare
College of Computing（计算机学院）；Umm Al-Qura University（乌姆·古拉大学）；King Khalid University（哈立德国王大学）；College of Computer and Information Sciences（计算机与信息科学学院）；King Saud University（沙特国王大学）
Newberry et al. (Detecting evolutionary forces in language change, Nature 551, 2017) tackle an important but difficult problem in linguistics, the testing of selective theories of language change against a null model of drift. Having applied a test from population genetics (the Frequency Increment Test) to a number of relevant examples, they suggest stochasticity has a previously under-appreciated role in language evolution. We replicate their results and find that while the overall observation holds, results produced by this approach on individual time series can be sensitive to how the corpu…
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Nanyang Technological University（南洋理工大学）
Cloze-style reading comprehension in Chinese is still limited due to the lack of various corpora. In this paper we propose a large-scale Chinese cloze test dataset ChID, which studies the comprehension of idiom, a unique language phenomenon in Chinese. In this corpus, the idioms in a passage are replaced by blank symbols and the correct answer needs to be chosen from well-designed candidate idioms. We carefully study how the design of candidate idioms and the representation of idioms affect the performance of state-of-the-art models. Results show that the machine accuracy is substantially wors…
Sofia University “St. Kliment Ohridski”（圣克莱门特·奥里季斯基索菲亚大学）；King’s College London（伦敦国王学院）；Qatar Computing Research Institute, HBKU（卡塔尔计算研究所（HBKU））；Sapienza University of Rome（罗马第一大学）；Facebook AI（脸书人工智能研究院）；University of Padova（帕多瓦大学）
Propaganda can be defined as a form of communication that aims to influence the opinions or the actions of people towards a specific goal; this is achieved by means of well-defined rhetorical and psychological devices. Propaganda, in the form we know it today, can be dated back to the beginning of the 17th century. However, it is with the advent of the Internet and the social media that it has started to spread on a much larger scale than before, thus becoming major societal and political issue. Nowadays, a large fraction of propaganda in social media is multimodal, mixing textual with visual …
Cloze tests are widely adopted in language exams to evaluate students' language proficiency. In this paper, we propose the first large-scale human-created cloze test dataset CLOTH, containing questions used in middle-school and high-school language exams. With missing blanks carefully created by teachers and candidate choices purposely designed to be nuanced, CLOTH requires a deeper language understanding and a wider attention span than previously automatically-generated cloze datasets. We test the performance of dedicatedly designed baseline models including a language model trained on the On…
KletterMix: Climbing Toward High-Quality German Pretraining Data - The Full Report
AI & ML Group, TU Darmstadt（人工智能与机器学习小组，德累斯顿技术大学）；Lab1141；Lamarr Institute（拉马尔研究所）；Fraunhofer IAIS（弗劳恩霍夫人工智能研究所）；hessian.AI（海斯坦.AI）；German Research Center for AI (DFKI)（德国人工智能研究中心（DFKI））；Centre for Cognitive Science, TU Darmstadt（认知科学中心，德累斯顿技术大学）
Identifying the quality of free-text arguments has become an important task in the rapidly expanding field of computational argumentation. In this work, we explore the challenging task of argument quality ranking. To this end, we created a corpus of 30,497 arguments carefully annotated for point-wise quality, released as part of this work. To the best of our knowledge, this is the largest dataset annotated for point-wise argument quality, larger by a factor of five than previously released datasets. Moreover, we address the core issue of inducing a labeled score from crowd annotations by perfo…
Mila-Quebec AI Institute（米拉-魁北克人工智能研究所）；Università Politecnica delle Marche（马尔凯理工大学）
In recent years, deep learning based source separation has achieved impressive results. Most studies, however, still evaluate separation models on synthetic datasets, while the performance of state-of-the-art techniques on in-the-wild speech data remains an open question. This paper contributes to fill this gap in two ways. First, we release the REAL-M dataset, a crowd-sourced corpus of real-life mixtures. Secondly, we address the problem of performance evaluation of real-life mixtures, where the ground truth is not available. We bypass this issue by carefully designing a blind Scale-Invariant…
BERT is a widely used pre-trained model in natural language processing. However, since BERT is quadratic to the text length, the BERT model is difficult to be used directly on the long-text corpus. In some fields, the collected text data may be quite long, such as in the health care field. Therefore, to apply the pre-trained language knowledge of BERT to long text, in this paper, imitating the skimming-intensive reading method used by humans when reading a long paragraph, the Skimming-Intensive Model (SkIn) is proposed. It can dynamically select the critical information in the text so that the…
University of Michigan（密歇根大学）；University of New Mexico（新墨西哥大学）；Emory University（埃默里大学）；University of Washington（华盛顿大学）；Allen Institute for AI（艾伦人工智能研究所）
Language models increasingly rely on massive web dumps for diverse text data. However, these sources are rife with undesirable content. As such, resources like Wikipedia, books, and newswire often serve as anchors for automatically selecting web text most suitable for language modeling, a process typically referred to as quality filtering. Using a new dataset of U.S. high school newspaper articles -- written by students from across the country -- we investigate whose language is preferred by the quality filter used for GPT-3. We find that newspapers from larger schools, located in wealthier, e…
Spoken medical dialogue systems are increasingly attracting interest to enhance access to healthcare services and improve quality and traceability of patient care. In this paper, we focus on medical drug prescriptions acquired on smartphones through spoken dialogue. Such systems would facilitate the traceability of care and would free clinicians' time. However, there is a lack of speech corpora to develop such systems since most of the related corpora are in text form and in English. To facilitate the research and development of spoken medical dialogue systems, we present, to the best of our k…
Towards Open Foundation Language Model and Corpus for Macedonian: A Low-Resource Language
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；Faculty of Electrical Engineering and Information Technologies, UKIM（圣基里尔麦托迪大学电气工程与信息技术学院）；Emteq Ltd.（Emteq有限公司）
mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus
Inria（法国国家信息与自动化技术研究院）；Département d’informatique de l’ENS, CNRS, PSL Research University（巴黎高等师范学院计算机科学系、国家科学研究中心、巴黎综合理工研究学院）；Institut du développement et des ressources en informatique scientifique, CNRS（科学信息发展与资源研究所、国家科学研究中心）；Sorbonne Université（索邦大学）；Common Crawl Foundation（Common Crawl基金会）；Université Paris-Saclay（巴黎萨克雷大学）
University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；School of Medicine and Public Health（医学与公共卫生学院）；Loyola University Chicago（芝加哥洛约拉大学）；Harvard Medical School（哈佛医学院）
Automatically summarizing patients' main problems from daily progress notes using natural language processing methods helps to battle against information and cognitive overload in hospital settings and potentially assists providers with computerized diagnostic decision support. Problem list summarization requires a model to understand, abstract, and generate clinical documentation. In this work, we propose a new NLP task that aims to generate a list of problems in a patient's daily care plan using input from the provider's progress notes during hospitalization. We investigate the performance o…
