In this paper, we detail our submission to the 2019, 7th year, BioASQ competition. We present our approach for Task-7b, Phase B, Exact Answering Task. These Question Answering (QA) tasks include Factoid, Yes/No, List Type Question answering. Our system is based on a contextual word embedding model. We have used a Bidirectional Encoder Representations from Transformers(BERT) based system, fined tuned for biomedical question answering task using BioBERT. In the third test batch set, our system achieved the highest MRR score for Factoid Question Answering task. Also, for List type question answering task our system achieved the highest recall score in the fourth test batch set. Along with our detailed approach, we present the results for our submissions, and also highlight identified downsides for our current approach and ways to improve them in our future experiments.
Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）；Barcelona Supercomputing Center（巴塞罗那超级计算中心）
Advancing the state-of-the-art in large-scale biomedical semantic indexing and question answering is the main focus of the BioASQ challenge. BioASQ organizes respective tasks where different teams develop systems that are evaluated on the same benchmark datasets that represent the real information needs of experts in the biomedical domain. This paper presents an overview of the ninth edition of the BioASQ challenge in the context of the Conference and Labs of the Evaluation Forum (CLEF) 2021. In this year, a new question answering task, named Synergy, is introduced to support researchers study…
National Center for Scientific Research “Demokritos”（“德谟克利特”国家科学研究中心）；Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）；National and Kapodistrian University of Athens（雅典国立卡波迪斯特里安大学）；Barcelona Supercomputing Center（巴塞罗那超级计算中心）
In this paper, we present an overview of the eighth edition of the BioASQ challenge, which ran as a lab in the Conference and Labs of the Evaluation Forum (CLEF) 2020. BioASQ is a series of challenges aiming at the promotion of systems and methodologies for large-scale biomedical semantic indexing and question answering. To this end, shared tasks are organized yearly since 2012, where different teams develop systems that compete on the same demanding benchmark datasets that represent the real information needs of experts in the biomedical domain. This year, the challenge has been extended with…
Overview of BioASQ 2022: The tenth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering
National Center for Scientific Research “Demokritos”（希腊德谟克利特国家科学研究中心）；Aristotle University of Thessaloniki（亚里士多德大学塞萨洛尼基分校）；Barcelona Supercomputing Center（巴塞罗那超级计算中心）
Overview of BioASQ 2023: The eleventh BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answering
National Center for Scientific Research “Demokritos”（希腊德谟克利特国家科学研究中心）；Aristotle University of Thessaloniki（亚里士多德大学）；Barcelona Supercomputing Center（巴塞罗那超级计算中心）
Overview of BioASQ 2026: The fourteenth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering
National Center for Scientific Research “Demokritos”（德谟克利特国家科学研究中心）；Barcelona Supercomputing Center（巴塞罗那超级计算中心）；Lomonosov Moscow State University（罗蒙诺索夫莫斯科国立大学）；Artificial Intelligence Research Institute（人工智能研究院）；HSE University（高等经济大学）；Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）
Overview of BioASQ 2025: The Thirteenth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering
Barcelona Supercomputing Center, Barcelona, Spain（巴塞罗那超级计算中心，巴塞罗那，西班牙）；Moscow State University, Russia（莫斯科国立大学，俄罗斯）；Artificial Intelligence Research Institute, Russia（人工智能研究 institute，俄罗斯）；Kazan Federal University, Russia（卡扎诺夫联邦大学，俄罗斯）；SberAI & Skoltech, Russia（SberAI 与 Skoltech，俄罗斯）；Aristotle University of Thessaloniki, Greece（亚里士多德大学，希腊）；Northwell Health, USA（Northwell Health，美国）；University of Padua, Italy（帕多瓦大学，意大利）；Archimedes, Athena Research Center, Greece（阿基米德，亚特兰蒂斯研究中心，希腊）
This paper presents Macquarie University's participation to the BioASQ Synergy Task, and BioASQ9b Phase B. In each of these tasks, our participation focused on the use of query-focused extractive summarisation to obtain the ideal answers to medical questions. The Synergy Task is an end-to-end question answering task on COVID-19 where systems are required to return relevant documents, snippets, and answers to a given question. Given the absence of training data, we used a query-focused summarisation system that was trained with the BioASQ8b training data set and we experimented with methods to …
This paper presents Macquarie University's participation to the two most recent BioASQ Synergy Tasks (as per June 2022), and to the BioASQ10 Task~B (BioASQ10b), Phase~B. In these tasks, participating systems are expected to generate complex answers to biomedical questions, where the answers may contain more than one sentence. We apply query-focused extractive summarisation techniques. In particular, we follow a sentence classification-based approach that scores each candidate sentence associated to a question, and the $n$ highest-scoring sentences are returned as the answer. The Synergy Task c…
In this paper, we present our work on the BioASQ pipeline. The goal is to answer four types of questions: summary, yes/no, factoids, and list. Our goal is to empirically evaluate different modules involved: the feature extractor and the sentence selection block. We used our pipeline to test the effectiveness of each module for all kinds of question types and perform error analysis. We defined metrics that are useful for future research related to the BioASQ pipeline critical to improve the performance of the training pipeline.
This paper describes our submission to the 2017 BioASQ challenge. We participated in Task B, Phase B which is concerned with biomedical question answering (QA). We focus on factoid and list question, using an extractive QA model, that is, we restrict our system to output substrings of the provided text snippets. At the core of our system, we use FastQA, a state-of-the-art neural QA system. We extended it with biomedical word embeddings and changed its answer layer to be able to answer list questions in addition to factoid questions. We pre-trained the model on a large-scale open-domain QA data…
Aristotle University of Thessaloniki（塞萨洛尼基亚里士多德大学）；National and Kapodistrian University of Athens（雅典国立卡波迪斯特里安大学）；University of Houston（休斯顿大学）
The results of the seventh edition of the BioASQ challenge are presented in this paper. The aim of the BioASQ challenge is the promotion of systems and methodologies through the organization of a challenge on the tasks of large-scale biomedical semantic indexing and question answering. In total, 30 teams with more than 100 systems participated in the challenge this year. As in previous years, the best systems were able to outperform the strong baselines. This suggests that state-of-the-art systems are continuously improving, pushing the frontier of research.
Task B Phase B of the 2019 BioASQ challenge focuses on biomedical question answering. Macquarie University's participation applies query-based multi-document extractive summarisation techniques to generate a multi-sentence answer given the question and the set of relevant snippets. In past participation we explored the use of regression approaches using deep learning architectures and a simple policy gradient architecture. For the 2019 challenge we experiment with the use of classification approaches with and without reinforcement learning. In addition, we conduct a correlation analysis betwee…
In this work, we describe our experiments and participating systems in the BioASQ Task 9b Phase B challenge of biomedical question answering. We have focused on finding the ideal answers and investigated multi-task fine-tuning and gradual unfreezing techniques on transformer-based language models. For factoid questions, our ALBERT-based systems ranked first in test batch 1 and fourth in test batch 2. Our DistilBERT systems outperformed the ALBERT variants in test batches 4 and 5 despite having 81% fewer parameters than ALBERT. However, we observed that gradual unfreezing had no significant imp…
University of Tennessee（田纳西大学）；Oak Ridge National Laboratory（橡树岭国家实验室）
Biomedical machine reading comprehension (biomedical-MRC) aims to comprehend complex biomedical narratives and assist healthcare professionals in retrieving information from them. The high performance of modern neural network-based MRC systems depends on high-quality, large-scale, human-annotated training datasets. In the biomedical domain, a crucial challenge in creating such datasets is the requirement for domain knowledge, inducing the scarcity of labeled data and the need for transfer learning from the labeled general-purpose (source) domain to the biomedical (target) domain. However, ther…
The amount of publicly available biomedical literature has been growing rapidly in recent years, yet question answering systems still struggle to exploit the full potential of this source of data. In a preliminary processing step, many question answering systems rely on retrieval models for identifying relevant documents and passages. This paper proposes a weighted cosine distance retrieval scheme based on neural network word embeddings. Our experiments are based on publicly available data and tasks from the BioASQ biomedical question answering challenge and demonstrate significant performance…
Factoid question answering (QA) has recently benefited from the development of deep learning (DL) systems. Neural network models outperform traditional approaches in domains where large datasets exist, such as SQuAD (ca. 100,000 questions) for Wikipedia articles. However, these systems have not yet been applied to QA in more specific domains, such as biomedicine, because datasets are generally too small to train a DL system from scratch. For example, the BioASQ dataset for biomedical QA comprises less then 900 factoid (single answer) and list (multiple answers) QA instances. In this work, we a…
In this work we present Google's submission to the BioASQ 7 biomedical question answering (QA) task (specifically Task 7b, Phase B). The core of our systems are based on BERT QA models, specifically the model of \cite{alberti2019bert}. In this report, and via our submissions, we aimed to investigate two research questions. We start by studying how domain portable are QA systems that have been pre-trained and fine-tuned on general texts, e.g., Wikipedia. We measure this via two submissions. The first is a non-adapted model that uses a public pre-trained BERT model and is fine-tuned on the Natur…
Information retrieval (IR) is essential in search engines and dialogue systems as well as natural language processing tasks such as open-domain question answering. IR serve an important function in the biomedical domain, where content and sources of scientific knowledge may evolve rapidly. Although neural retrievers have surpassed traditional IR approaches such as TF-IDF and BM25 in standard open-domain question answering tasks, they are still found lacking in the biomedical domain. In this paper, we seek to improve information retrieval (IR) using neural retrievers (NR) in the biomedical doma…
University of Massachusetts Amherst, USA（马萨诸塞大学阿默斯特分校（美国））；Chan Zuckerberg Initiative, USA（陈-扎克伯格倡议组织（美国））
We explore the suitability of unsupervised representation learning methods on biomedical text -- BioBERT, SciBERT, and BioSentVec -- for biomedical question answering. To further improve unsupervised representations for biomedical QA, we introduce a new pre-training task from unlabeled data designed to reason about biomedical entities in the context. Our pre-training method consists of corrupting a given context by randomly replacing some mention of a biomedical entity with a random entity mention and then querying the model with the correct entity mention in order to locate the corrupted part…
Information Retrieval (IR) is the task of obtaining pieces of data (such as documents) that are relevant to a particular query or need from a large repository of information. IR is a valuable component of several downstream Natural Language Processing (NLP) tasks. Practically, IR is at the heart of many widely-used technologies like search engines. While probabilistic ranking functions like the Okapi BM25 function have been utilized in IR systems since the 1970's, modern neural approaches pose certain advantages compared to their classical counterparts. In particular, the release of BERT (Bidi…
Athens University of Economics and Business（雅典经济与商业大学）；Institute for Language and Speech Processing, Research Center ‘Athena’（“雅典娜”研究中心语言与语音处理研究所）；Institute of Informatics and Telecommunications, NCSR ‘Demokritos’（国家科学研究中心“德谟克利特”信息与电信研究所）
We study the effect of seven data augmentation (da) methods in factoid question answering, focusing on the biomedical domain, where obtaining training instances is particularly difficult. We experiment with data from the BioASQ challenge, which we augment with training instances obtained from an artificial biomedical machine reading comprehension dataset, or via back-translation, information retrieval, word substitution based on word2vec embeddings, or masked language modeling, question generation, or extending the given passage with additional context. We show that da can lead to very signifi…
The overwhelming amount of biomedical scientific texts calls for the development of effective language models able to tackle a wide range of biomedical natural language processing (NLP) tasks. The most recent dominant approaches are domain-specific models, initialized with general-domain textual data and then trained on a variety of scientific corpora. However, it has been observed that for specialized domains in which large corpora exist, training a model from scratch with just in-domain knowledge may yield better results. Moreover, the increasing focus on the compute costs for pre-training r…
Information Retrieval (IR) is the task of obtaining pieces of data (such as documents or snippets of text) that are relevant to a particular query or need from a large repository of information. While a combination of traditional keyword- and modern BERT-based approaches have been shown to be effective in recent work, there are often nuances in identifying what information is "relevant" to a particular query, which can be difficult to properly capture using these systems. This work introduces the concept of a Multi-Perspective IR system, a novel methodology that combines multiple deep learning…
Using deep learning models on small scale datasets would result in overfitting. To overcome this problem, the process of pre-training a model and fine-tuning it to the small scale dataset has been used extensively in domains such as image processing. Similarly for question answering, pre-training and fine-tuning can be done in several ways. Commonly reading comprehension models are used for pre-training, but we show that other types of pre-training can work better. We compare two pre-training models based on reading comprehension and open domain question answering models and determine the perf…
Korea University（高丽大学）；AstraZeneca UK（阿斯利康英国分公司）；AstraZeneca SE（阿斯利康瑞典分公司）；AIGEN Sciences Inc.（AIGEN科学公司）
Current studies in extractive question answering (EQA) have modeled the single-span extraction setting, where a single answer span is a label to predict for a given question-passage pair. This setting is natural for general domain EQA as the majority of the questions in the general domain can be answered with a single span. Following general domain EQA models, current biomedical EQA (BioEQA) models utilize the single-span extraction setting with post-processing steps. In this article, we investigate the question distribution across the general and biomedical domains and discover biomedical que…
With the large and increasing volume of textual data, automated methods for identifying significant topics to classify textual documents have received a growing interest. While many efforts have been made in this direction, it still remains a real challenge. Moreover, the issue is even more complex as full texts are not always freely available. Then, using only partial information to annotate these documents is promising but remains a very ambitious issue. MethodsWe propose two classification methods: a k-nearest neighbours (kNN)-based approach and an explicit semantic analysis (ESA)-based app…
Korea University（高丽大学）；Hyundai Motor Company（现代汽车公司）
Biomedical question answering (QA) is a challenging task due to the scarcity of data and the requirement of domain expertise. Pre-trained language models have been used to address these issues. Recently, learning relationships between sentence pairs has been proved to improve performance in general QA. In this paper, we focus on applying BioBERT to transfer the knowledge of natural language inference (NLI) to biomedical QA. We observe that BioBERT trained on the NLI dataset obtains better performance on Yes/No (+5.59%), Factoid (+0.53%), List type (+13.58%) questions compared to performance ob…
Harnessing Collective Intelligence of LLMs for Robust Biomedical QA: A Multi-Model Approach
Department of Informatics；Kapodistrian University of Athens, Greece；Institute for Fundamental Biomedical Science, Biomedical Sciences Research Center "Alexander Fleming", Greece；Department of Informatics \& Telematics, School of Digital Technology, Harokopio University, Greece；Archimedes, Athena Research Center, Greece
This paper describes our system, dubbed WS4A (Web Services for All), that participated in the fourth edition of the BioASQ challenge (2016). We used WS4A to perform the Question and Answering (QA) task 4b, which consisted on the retrieval of relevant concepts, documents, snippets, RDF triples, exact answers and ideal answers for each given question. The novelty in our approach consists on the maximum exploitation of existing web services in each step of WS4A, such as the annotation of text, and the retrieval of metadata for each annotation. The information retrieved included concept identifier…
Think Parallax: Solving Multi-Hop Problems via Multi-View Knowledge-Graph-Based Retrieval-Augmented Generation
Yangtze Delta Region Institute (Huzhou), University of Electronic Science and Technology of China（长江三角洲地区研究院（湖州），电子科学与技术大学）；School of Information and Software Engineering, University of Electronic Science and Technology of China（信息与软件工程学院，电子科学与技术大学）
The recent success of question answering systems is largely attributed to pre-trained language models. However, as language models are mostly pre-trained on general domain corpora such as Wikipedia, they often have difficulty in understanding biomedical questions. In this paper, we investigate the performance of BioBERT, a pre-trained biomedical language model, in answering biomedical questions including factoid, list, and yes/no type questions. BioBERT uses almost the same structure across various question types and achieved the best performance in the 7th BioASQ Challenge (Task 7b, Phase B).…
Comparative Analysis of 47 Context-Based Question Answer Models Across 8 Diverse Datasets
School of Chemistry and Molecular Biology, The University of Queensland（化学与分子生物学学院，昆士兰大学）；Computational Biology and Clinical Informatics, Baker Heart and Diabetes Institute（计算生物学与临床信息学，贝克心脏病与糖尿病研究所）；Mechanical Engineering Department, Khalifa University（机械工程系，卡利法大学）
Using Pretrained Large Language Model with Prompt Engineering to Answer Biomedical Questions
Georgia Institute of Technology（佐治亚理工学院）；Peoples' Friendship University of Russia (RUDN University)（俄罗斯人民友谊大学（RUDN大学））；Joint Institute for Nuclear Research（联合核研究所）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Skövde（舍夫德大学）
CaresAI at BioCreative IX Track 1 -- LLM for Biomedical QA
Cairo University, Faculty of Engineering, Computer Engineering Department（开罗大学工程学院计算机工程系）；Menzies School of Health Research, Charles Darwin University, NT, Australia（梅恩兹健康研究学院，查尔斯达尔文大学，澳大利亚NT）；Department of Biology, Boston College, Massachusetts, USA（生物学系，波士顿学院，马萨诸塞州，美国）；Peoples' Friendship University of Russia (RUDN University)（俄罗斯人民友谊大学（RUDN大学））；Joint Institute for Nuclear Research（联合核研究中心）；Vrije Universiteit Amsterdam（阿姆斯特丹自由大学）；University of Skövde（斯德哥尔摩大学）
