arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2401.08104cs.IR
收藏

Goldilocks 的可复现性研究:为 TAR 恰到好处地微调 BERT

A Reproducibility Study of Goldilocks: Just-Right Tuning of BERT for TAR

Xinyu Mao, Bevan Koopman, Guido Zuccon

AI总结:

本文复现并验证了 Goldilocks 研究中进一步预训练对 BERT 主动学习 TAR 流程的关键作用,并进一步发现在医学系统综述等特定领域任务中使用领域专用 BERT 骨干网络时无需额外预训练,为领域 TAR 应用提供了实践指导。

摘要:

文档筛选是高召回率检索任务中繁琐且耗时的一环,例如编写系统性文献综述时,目标是识别出与某主题相关的所有文档。为了帮助简化这一过程,许多技术辅助评审(TAR)方法利用主动学习技术来减少需要人工评审的文档数量。基于 BERT 的模型在文本分类中已展现出高效能,这引发了人们对其在 TAR 工作流中潜在应用价值的兴趣。本文研究了近期一项考察进一步预训练轮数对基于 BERT 的主动学习流程有效性和效率影响的工作。我们首先报告,我们能够在两个特定的 TAR 数据集上复现原始实验,并证实了部分发现:重要的是,进一步预训练对于实现高效能至关重要,但在选择正确的训练轮数方面需要加以注意。随后,我们研究了该流程在不同 TAR 任务——医学系统性综述——上的泛化能力。在这一背景下,我们表明,如果在主动学习流程中使用领域特定的 BERT 骨干网络,则无需进一步预训练。这一发现为在领域特定的 TAR 任务中使用所研究的主动学习流程提供了实践启示。

英文摘要:

Screening documents is a tedious and time-consuming aspect of high-recall retrieval tasks, such as compiling a systematic literature review, where the goal is to identify all relevant documents for a topic. To help streamline this process, many Technology-Assisted Review (TAR) methods leverage active learning techniques to reduce the number of documents requiring review. BERT-based models have shown high effectiveness in text classification, leading to interest in their potential use in TAR workflows. In this paper, we investigate recent work that examined the impact of further pre-training epochs on the effectiveness and efficiency of a BERT-based active learning pipeline. We first report that we could replicate the original experiments on two specific TAR datasets, confirming some of the findings: importantly, that further pre-training is critical to high effectiveness, but requires attention in terms of selecting the correct training epoch. We then investigate the generalisability of the pipeline on a different TAR task, that of medical systematic reviews. In this context, we show that there is no need for further pre-training if a domain-specific BERT backbone is used within the active learning pipeline. This finding provides practical implications for using the studied active learning pipeline within domain-specific TAR tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑