首页/论文/2005.04474

arXiv 2005.044742020-05-09cs.IR

TREC-COVID: Constructing a Pandemic Information Retrieval Test Collection

Ellen Voorhees, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, William R Hersh, Kyle Lo, Kirk Roberts, Ian Soboroff, Lucy Lu Wang

备注：: 10 pages, 5 figures. TREC-COVID web site: http://ir.nist.gov/covidSubmit/ Will also appear in June 2020 issue of ACM SIGIR Forum

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

TREC-COVID is a community evaluation designed to build a test collection that captures the information needs of biomedical researchers using the scientific literature during a pandemic. One of the key characteristics of pandemic search is the accelerated rate of change: the topics of interest evolve as the pandemic progresses and the scientific literature in the area explodes. The COVID-19 pandemic provides an opportunity to capture this progression as it happens. TREC-COVID, in creating a test collection around COVID-19 literature, is building infrastructure to support new research and technologies in pandemic search.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Participation in TREC 2020 COVID Track Using Continuous Active Learning

We describe our participation in all five rounds of the TREC 2020 COVID Track (TREC-COVID). The goal of TREC-COVID is to contribute to the response to the COVID-19 pandemic by identifying answers to many pressing questions and building infrastructure to improve search systems [8]. All five rounds of this Track challenged participants to perform a classic ad-hoc search task on the new data collection CORD-19. Our solution addressed this challenge by applying the Continuous Active Learning model (CAL) and its variations. Our results showed us to be amongst the top scoring manual runs and we rema…

2020-11-03cs.IR

Searching for Scientific Evidence in a Pandemic: An Overview of TREC-COVID

We present an overview of the TREC-COVID Challenge, an information retrieval (IR) shared task to evaluate search on scientific literature related to COVID-19. The goals of TREC-COVID include the construction of a pandemic search test collection and the evaluation of IR methods for COVID-19. The challenge was conducted over five rounds from April to July, 2020, with participation from 92 unique teams and 556 individual submissions. A total of 50 topics (sets of related queries) were used in the evaluation, starting at 30 topics for Round 1 and adding 5 new topics per round to target emerging to…

2021-04-19cs.IR

RMITB at TREC COVID 2020

Search engine users rarely express an information need using the same query, and small differences in queries can lead to very different result sets. These user query variations have been exploited in past TREC CORE tracks to contribute diverse, highly-effective runs in offline evaluation campaigns with the goal of producing reusable test collections. In this paper, we document the query fusion runs submitted to the first and second round of TREC COVID, using ten queries per topic created by the first author. In our analysis, we focus primarily on the effects of having our second priority run …

2020-11-09cs.IR

Exploring task-based query expansion at the TREC-COVID track

We explore how to generate effective queries based on search tasks. Our approach has three main steps: 1) identify search tasks based on research goals, 2) manually classify search queries according to those tasks, and 3) compare three methods to improve search rankings based on the task context. The most promising approach is based on expanding the user's query terms using task terms, which slightly improved the NDCG@20 scores over a BM25 baseline. Further improvements might be gained if we can identify more specific search tasks.

2020-11-16cs.IR

RRF102: Meeting the TREC-COVID Challenge with a 100+ Runs Ensemble

Google Research（谷歌研究院）；Alibaba Group（阿里巴巴集团）

In this paper, we report the results of our participation in the TREC-COVID challenge. To meet the challenge of building a search engine for rapidly evolving biomedical collection, we propose a simple yet effective weighted hierarchical rank fusion approach, that ensembles together 102 runs from (a) lexical and semantic retrieval systems, (b) pre-trained and fine-tuned BERT rankers, and (c) relevance feedback runs. Our ablation studies demonstrate the contributions of each of these systems to the overall ensemble. The submitted ensemble runs achieved state-of-the-art performance in rounds 4 an…

2020-10-01cs.CLcs.IR

Denmark's Participation in the Search Engine TREC COVID-19 Challenge: Lessons Learned about Searching for Precise Biomedical Scientific Information on COVID-19

University of Copenhagen（哥本哈根大学）；Aalborg University（奥尔堡大学）

This report describes the participation of two Danish universities, University of Copenhagen and Aalborg University, in the international search engine competition on COVID-19 (the 2020 TREC-COVID Challenge) organised by the U.S. National Institute of Standards and Technology (NIST) and its Text Retrieval Conference (TREC) division. The aim of the competition was to find the best search engine strategy for retrieving precise biomedical scientific information on COVID-19 from the largest, at that point in time, dataset of curated scientific literature on COVID-19 -- the COVID-19 Open Research D…

2020-11-26cs.IRcs.LG

更多相关论文收起更多论文

CMT in TREC-COVID Round 2: Mitigating the Generalization Gaps from Web to Special Domain Search

Tsinghua University（清华大学）；Microsoft Research（微软研究院）；Carnegie Mellon University（卡内基梅隆大学）

Neural rankers based on deep pretrained language models (LMs) have been shown to improve many information retrieval benchmarks. However, these methods are affected by their the correlation between pretraining domain and target domain and rely on massive fine-tuning relevance labels. Directly applying pretraining methods to specific domains may result in suboptimal search quality because specific domains may have domain adaption problems, such as the COVID domain. This paper presents a search system to alleviate the special domain adaption problem. The system utilizes the domain-adaptive pretra…

2020-11-03cs.CLcs.IR

An Evaluation of Two Commercial Deep Learning-Based Information Retrieval Systems for COVID-19 Literature

The COVID-19 pandemic has resulted in a tremendous need for access to the latest scientific information, primarily through the use of text mining and search tools. This has led to both corpora for biomedical articles related to COVID-19 (such as the CORD-19 corpus (Wang et al., 2020)) as well as search engines to query such data. While most research in search engines is performed in the academic field of information retrieval (IR), most academic search engines$\unicode{x2013}$though rigorously evaluated$\unicode{x2013}$are sparsely utilized, while major commercial web search engines (e.g., Goo…

2020-07-27cs.IR

Repurposing TREC-COVID Annotations to Answer the Key Questions of CORD-19

Pennsylvania State University（宾夕法尼亚州立大学）

The novel coronavirus disease 2019 (COVID-19) began in Wuhan, China in late 2019 and to date has infected over 14M people worldwide, resulting in over 750,000 deaths. On March 10, 2020 the World Health Organization (WHO) declared the outbreak a global pandemic. Many academics and researchers, not restricted to the medical domain, began publishing papers describing new discoveries. However, with the large influx of publications, it was hard for these individuals to sift through the large amount of data and make sense of the findings. The White House and a group of industry research labs, lead b…

2020-08-27cs.CLcs.IRcs.LG

IR模型与COVID-19大流行：性能与挑战的比较研究

IR Models and the COVID-19 Pandemic: A Comparative Study of Performance and Challenges

本研究基于CORD-19数据集和TREC框架评估了多种IR模型，发现BERT和Contriever等高级模型在疫情期间检索相关信息表现更好，强调了定制化IR系统应对信息过载的重要性。

2023-05-21cs.IR

Searching Scientific Literature for Answers on COVID-19 Questions

CSIRO Data61（澳大利亚联邦科学与工业研究组织 Data61）；The Australian National University（澳大利亚国立大学）

Finding answers related to a pandemic of a novel disease raises new challenges for information seeking and retrieval, as the new information becomes available gradually. TREC COVID search track aims to assist in creating search tools to aid scientists, clinicians, policy makers and others with similar information needs in finding reliable answers from the scientific literature. We experiment with different ranking algorithms as part of our participation in this challenge. We propose a novel method for neural retrieval, and demonstrate its effectiveness on the TREC COVID search.

2020-07-06cs.IRcs.LG

MeSH descriptors indicate the knowledge growth in the SARS-CoV-2/COVID-19 pandemic

The scientific papers dealing with the novel betacoronavirus SARS-CoV-2 and the coronavirus disease 2019 (COVID-19) caused by this virus, published in 2020 and recorded in the database PUBMED, were retrieved on April 27, 2020. About 20\% of the records contain Medical Subject Headings (MeSH), keywords assigned to records in the course of the indexing process in order to summarise the articles' contents. The temporal sequence of the first occurrences of the keywords was determined, thus giving insight into the growth of the knowledge base of the pandemic.

2020-05-13cs.DLcs.IR

SLEDGE-Z: A Zero-Shot Baseline for COVID-19 Literature Search

Georgetown University（乔治城大学）；Allen Institute for AI（艾伦人工智能研究所）

With worldwide concerns surrounding the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), there is a rapidly growing body of scientific literature on the virus. Clinicians, researchers, and policy-makers need to be able to search these articles effectively. In this work, we present a zero-shot ranking algorithm that adapts to COVID-related scientific literature. Our approach filters training data from another collection down to medical-related queries, uses a neural re-ranking model pre-trained on scientific text (SciBERT), and filters the target document collection. This approach …

2020-10-12cs.CLcs.IR

Artificial Intelligence (AI) in Action: Addressing the COVID-19 Pandemic with Natural Language Processing (NLP)

National Center for Biotechnology Information（国家生物技术信息中心）；National Library of Medicine（国家医学图书馆）；National Institutes of Health（国立卫生研究院）

The COVID-19 pandemic has had a significant impact on society, both because of the serious health effects of COVID-19 and because of public health measures implemented to slow its spread. Many of these difficulties are fundamentally information needs; attempts to address these needs have caused an information overload for both researchers and the public. Natural language processing (NLP), the branch of artificial intelligence that interprets human language, can be applied to address many of the information needs made urgent by the COVID-19 pandemic. This review surveys approximately 150 NLP st…

2021-09-05cs.CLcs.IRcs.LG

Exploring celebrity influence on public attitude towards the COVID-19 pandemic: social media shared sentiment analysis

The COVID-19 pandemic has introduced new opportunities for health communication, including an increase in the public use of online outlets for health-related emotions. People have turned to social media networks to share sentiments related to the impacts of the COVID-19 pandemic. In this paper we examine the role of social messaging shared by Persons in the Public Eye (i.e. athletes, politicians, news personnel) in determining overall public discourse direction. We harvested approximately 13 million tweets ranging from 1 January 2020 to 1 March 2022. The sentiment was calculated for each tweet…

2023-02-23cs.CLcs.IRcs.LG