arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.16636cs.IRcs.CLcs.DL
收藏

DataFinder:基于自然语言描述的科学数据集推荐

DataFinder: Scientific Dataset Recommendation from Natural Language Descriptions

Carnegie Mellon University(卡内基梅隆大学);Shanghai Jiao Tong University(上海交通大学);Inspired Cognition(Inspired Cognition公司)

Vijay Viswanathan, Luyu Gao, Tongshuang Wu, Pengfei Liu, Graham Neubig

AI总结:

本文提出根据自然语言研究描述推荐科学数据集的任务,构建 DataFinder Dataset,并训练优于现有第三方搜索引擎的双编码器检索器。

摘要:

现代机器学习依赖数据集来发展和验证研究想法。随着公开可用数据的增长,找到合适的数据集变得越来越困难。任何研究问题都会对某一数据集能在多大程度上帮助研究者回答该问题施加显式和隐式约束,例如数据集规模、模态和领域。我们将根据研究想法的简短自然语言描述推荐数据集这一任务进行操作化定义,以帮助人们找到满足需求的相关数据集。作为一个信息检索问题,数据集推荐面临独特挑战:数据集难以直接为搜索建立索引,而且没有可直接用于该任务的语料库。为促进这一任务,我们构建了 DataFinder Dataset,其中包含一个较大的自动构建训练集(17.5K 个查询)和一个较小的专家标注评估集(392 个查询)。利用这些数据,我们在测试集上比较了多种信息检索算法,并提出了一种用于基于文本的数据集推荐的、性能更优的双编码器检索器。该系统在 DataFinder Dataset 上训练,相比现有第三方数据集搜索引擎能够找到更多相关搜索结果。为鼓励数据集推荐方向的进展,我们向公众发布了数据集和模型。

英文摘要:

Modern machine learning relies on datasets to develop and validate research ideas. Given the growth of publicly available data, finding the right dataset to use is increasingly difficult. Any research question imposes explicit and implicit constraints on how well a given dataset will enable researchers to answer this question, such as dataset size, modality, and domain. We operationalize the task of recommending datasets given a short natural language description of a research idea, to help people find relevant datasets for their needs. Dataset recommendation poses unique challenges as an information retrieval problem; datasets are hard to directly index for search and there are no corpora readily available for this task. To facilitate this task, we build the DataFinder Dataset which consists of a larger automatically-constructed training set (17.5K queries) and a smaller expert-annotated evaluation set (392 queries). Using this data, we compare various information retrieval algorithms on our test set and present a superior bi-encoder retriever for text-based dataset recommendation. This system, trained on the DataFinder Dataset, finds more relevant search results than existing third-party dataset search engines. To encourage progress on dataset recommendation, we release our dataset and models to the public.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑