Electronic medical records (EMR) contain comprehensive patient information and are typically stored in a relational database with multiple tables. Effective and efficient patient information retrieval from EMR data is a challenging task for medical experts. Question-to-SQL generation methods tackle this problem by first predicting the SQL query for a given question about a database, and then, executing the query on the database. However, most of the existing approaches have not been adapted to the healthcare domain due to a lack of healthcare Question-to-SQL dataset for learning models specific to this domain. In addition, wide use of the abbreviation of terminologies and possible typos in questions introduce additional challenges for accurately generating the corresponding SQL queries. In this paper, we tackle these challenges by developing a deep learning based TRanslate-Edit Model for Question-to-SQL (TREQS) generation, which adapts the widely used sequence-to-sequence model to directly generate the SQL query for a given question, and further performs the required edits using an attentive-copying mechanism and task-specific look-up tables. Based on the widely used publicly available electronic medical database, we create a new large-scale Question-SQL pair dataset, named MIMICSQL, in order to perform the Question-to-SQL generation task in healthcare domain. An extensive set of experiments are conducted to evaluate the performance of our proposed model on MIMICSQL. Both quantitative and qualitative experimental results indicate the flexibility and efficiency of our proposed method in predicting condition values and its robustness to random questions with abbreviations and typos.
Universidade de São Paulo（圣保罗大学）；Center for Artificial Intelligence (C4AI)（人工智能中心（C4AI））
Deep learning transformers have drastically improved systems that automatically answer questions in natural language. However, different questions demand different answering techniques; here we propose, build and validate an architecture that integrates different modules to answer two distinct kinds of queries. Our architecture takes a free-form natural language text and classifies it to send it either to a Neural Question Answering Reasoner or a Natural Language parser to SQL. We implemented a complete system for the Portuguese language, using some of the main tools available for the language…
SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA
Nanyang Technological University（南洋理工大学）；NYU Shanghai（上海纽约大学）；Alibaba-NTU Singapore Joint Research Institute（阿里巴巴-南洋理工大学新加坡联合研究院）；Center for Data Science, New York University（纽约大学数据科学中心）
In this paper, we describe a dataset and baseline result for a question answering that utilizes web tables. It contains commonly asked questions on the web and their corresponding answers found in tables on websites. Our dataset is novel in that every question is paired with a table of a different signature. In particular, the dataset contains two classes of tables: entity-instance tables and the key-value tables. Each QA instance comprises a table of either kind, a natural language question, and a corresponding structured SQL query. We build our model by dividing question answering into sever…
The current state-of-the-art generative models for open-domain question answering (ODQA) have focused on generating direct answers from unstructured textual information. However, a large amount of world's knowledge is stored in structured databases, and need to be accessed using query languages such as SQL. Furthermore, query languages can answer questions that require complex reasoning, as well as offering full explainability. In this paper, we propose a hybrid framework that takes both textual and tabular evidence as input and generates either direct answers or SQL queries depending on which…
University of Bonn（波恩大学）；Ruhr University Bochum（波鸿鲁尔大学）
Translating natural language to SQL queries for table-based question answering is a challenging problem and has received significant attention from the research community. In this work, we extend a pointer-generator and investigate the order-matters problem in semantic parsing for SQL. Even though our model is a straightforward extension of a general-purpose pointer-generator, it outperforms early works for WikiSQL and remains competitive to concurrently introduced, more complex models. Moreover, we provide a deeper investigation of the potential order-matters problem that could arise due to h…
An intelligent machine that can answer human questions based on electronic health records (EHR-QA) has a great practical value, such as supporting clinical decisions, managing hospital administration, and medical chatbots. Previous table-based QA studies focusing on translating natural questions into table queries (NLQ2SQL), however, suffer from the unique nature of EHR data due to complex and specialized medical terminology, hence increased decoding difficulty. In this paper, we design UniQA, a unified encoder-decoder architecture for EHR-QA where natural language questions are converted to q…
University of Washington（华盛顿大学）；Microsoft Research（微软研究院）
The goal of database question answering is to enable natural language querying of real-life relational databases in diverse application domains. Recently, large-scale datasets such as Spider and WikiSQL facilitated novel modeling techniques for text-to-SQL parsing, improving zero-shot generalization to unseen databases. In this work, we examine the challenges that still prevent these techniques from practical deployment. First, we present KaggleDBQA, a new cross-domain evaluation dataset of real Web databases, with domain-specific data types, original formatting, and unrestricted questions. Se…
Evaluating and Enhancing LLMs for Multi-turn Text-to-SQL with Multiple Question Types
School of Computer Science, Harbin University of Science and Technology（哈尔滨理工大学计算机科学学院）；Faculty of Computing, Harbin Institute of Technology（哈尔滨工业大学计算学部）
Rokid AI Lab（Rokid人工智能实验室）；China Electronic Technology Group Corporation Information Science Academy（中国电子科技集团公司信息科学研究院）
Semantic parsing is the task of mapping natural language to logic form. In question answering, semantic parsing can be used to map the question to logic form and execute the logic form to get the answer. One key problem for semantic parsing is the hard label work. We study this problem in another way: we do not use the logic form any more. Instead we only use the schema and answer info. We think that the logic form step can be injected into the deep model. The reason why we think removing the logic form step is possible is that human can do the task without explicit logic form. We use BERT-bas…
Toward Auditable Neuro-Symbolic Reasoning in Pathology: SQL as an Explicit Trace of Evidence
Capital Normal University, Beijing, China（首都师范大学）；Leibniz-Institut für Analytische Wissenschaften-ISAS, Dortmund, Germany（莱比锡分析科学研究所-ISAS）；Harbin Institute of Technology, Shenzhen, China（哈尔滨工业大学深圳校区）
BiomedSQL: Text-to-SQL for Scientific Reasoning on Biomedical Knowledge Bases
Center for Alzheimer’s and Related Dementias, NIA, NIH（阿尔茨海默病及相关痴呆症研究中心，国家老龄化研究所，国立卫生研究院）；DataTecnica LLC（DataTecnica公司）；Johns Hopkins University（约翰霍普金斯大学）；Laboratory of Neurogenetics, NIA, NIH（神经遗传学实验室，国家老龄化研究所，国立卫生研究院）
University of Florida（佛罗里达大学）；The University of Texas Health Science Center at Houston（德克萨斯大学休斯顿健康科学中心）
This paper develops the first question answering dataset (DrugEHRQA) containing question-answer pairs from both structured tables and unstructured notes from a publicly available Electronic Health Record (EHR). EHRs contain patient records, stored in structured tables and unstructured clinical notes. The information in structured and unstructured EHRs is not strictly disjoint: information may be duplicated, contradictory, or provide additional context between these sources. Our dataset has medication-related queries, containing over 70,000 question-answer pairs. To provide a baseline model and…
Question Answering on Electronic Health Records (EHR-QA) has a significant impact on the healthcare domain, and it is being actively studied. Previous research on structured EHR-QA focuses on converting natural language queries into query language such as SQL or SPARQL (NLQ2Query), so the problem scope is limited to pre-defined data types by the specific query language. In order to expand the EHR-QA task beyond this limitation to handle multi-modal medical data and solve complex inference in the future, more primitive systemic language is needed. In this paper, we design the program-based mode…
Our research is in the relatively unexplored area of question answering technologies for patient-specific questions over their electronic health records. A large dataset of human expert curated question and answer pairs is an important pre-requisite for developing, training and evaluating any question answering system that is powered by machine learning. In this paper, we describe a process for creating such a dataset of questions and answers. Our methodology is replicable, can be conducted by medical students as annotators, and results in high inter-annotator agreement (0.71 Cohen's kappa). O…
ODTQA-FoRe: An Open-Domain Tabular Question Answering Dataset for Future Data Forecasting and Reasoning
School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；Institute of Artificial Intelligence and Future Networks, Beijing Normal University（北京师范大学人工智能与未来网络研究院）；Faculty of Arts and Sciences, Beijing Normal University（北京师范大学文理学院）；Beijing Normal-Hong Kong Baptist University（北京师范大学-香港 Baptist大学）
Can Language Models Analyze Data? Evaluating Large Language Models for Question Answering over Datasets
School of Production Engineering and Management, Technical University of Crete（生产工程与管理学院，希腊克里特技术大学）；Institute of Computer Science, Foundation for Research and Technology - Hellas（计算机科学研究所，希腊基础研究与技术研究院）
MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）；IBM TJ Watson Research Center（IBM托马斯·J·沃森研究中心）；University of Illinois Urbana Champaign（伊利诺伊大学厄巴纳-香槟分校）；Carle Illinois College of Medicine（伊利诺伊大学卡勒医学院）
We propose a novel methodology to generate domain-specific large-scale question answering (QA) datasets by re-purposing existing annotations for other NLP tasks. We demonstrate an instance of this methodology in generating a large-scale QA dataset for electronic medical records by leveraging existing expert annotations on clinical notes for various NLP tasks from the community shared i2b2 datasets. The resulting corpus (emrQA) has 1 million question-logical form and 400,000+ question-answer evidence pairs. We characterize the dataset and explore its learning potential by training baseline mode…
Structured queries expressed in languages (such as SQL, SPARQL, or XQuery) offer a convenient and explicit way for users to express their information needs for a number of tasks. In this work, we present an approach to answer these directly over text data without storing results in a database. We specifically look at the case of knowledge bases where queries are over entities and the relations between them. Our approach combines distributed query answering (e.g. Triple Pattern Fragments) with models built for extractive question answering. Importantly, by applying distributed querying answerin…
The Ohio State University（俄亥俄州立大学）；Nanjing University（南京大学）
Recent advances in deep learning have greatly propelled the research on semantic parsing. Improvement has since been made in many downstream tasks, including natural language interface to web APIs, text-to-SQL generation, among others. However, despite the close connection shared with these tasks, research on question answering over knowledge bases (KBQA) has comparatively been progressing slowly. We identify and attribute this to two unique challenges of KBQA, schema-level complexity and fact-level complexity. In this survey, we situate KBQA in the broader literature of semantic parsing and g…
Multi-modal Record Linkage is the process of matching multi-modal records from multiple sources that represent the same entity. This field has not been explored in research and we propose two solutions based on Deep Learning architectures that are inspired by recent work in Visual Question Answering. The neural networks we propose use two different fusion modules, the Recurrent Neural Network + Convolutional Neural Network fusion module and the Stacked Attention Network fusion module, that jointly combine the visual and the textual data of the records. The output of these fusion models is the …
HypEHR: Hyperbolic Modeling of Electronic Health Records for Efficient Question Answering
Department of Computer Science, Stony Brook University（石英布大学计算机科学系）；Department of Data Science, New Jersey Institute of Technology（新泽西理工学院数据科学系）；Department of Biomedical Informatics, Stony Brook University（石英布大学生物医学信息学系）
Question Answering (QA) is a widely-used framework for developing and evaluating an intelligent machine. In this light, QA on Electronic Health Records (EHR), namely EHR QA, can work as a crucial milestone towards developing an intelligent agent in healthcare. EHR data are typically stored in a relational database, which can also be converted to a directed acyclic graph, allowing two approaches for EHR QA: Table-based QA and Knowledge Graph-based QA. We hypothesize that the graph-based approach is more suitable for EHR QA as graphs can represent relations between entities and values more natur…
EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Images
KAIST（韩国科学技术院）；Microsoft Research Asia（微软亚洲研究院）；Centre of Perceptual Interactive Intelligence（感知交互智能中心）；Seoul National University Bundang Hospital（首尔大学盆唐医院）
