CSDR-BERT:一个用于中文科学数据集检索的预训练科学数据集匹配模型
CSDR-BERT: a pre-trained scientific dataset match model for Chinese Scientific Dataset Retrieval
摘要:
在开放科学运动的推动下,互联网上开放和共享的科学数据集越来越多,有效地检索这些数据集是信息检索研究的一项重要任务。近年来,大模型的发展,特别是预训练和微调范式的发展,为IR匹配任务提供了新的解决方案,该范式涉及对大模型的预训练和对下游任务的微调。本文在嵌入层使用原有的BERT标记,在模型层引入SimCSE和K-最近邻方法改进句子-BERT模型结构,在优化阶段使用余弦损失函数优化目标输出。我们的实验结果表明,我们的模型优于其他竞争模型在公共和自建数据集通过比较实验和消融实现。本研究探索并验证了中文科学数据集语义检索预训练技术的可行性和有效性。
英文摘要:
As the number of open and shared scientific datasets on the Internet increases under the open science movement, efficiently retrieving these datasets is a crucial task in information retrieval (IR) research. In recent years, the development of large models, particularly the pre-training and fine-tuning paradigm, which involves pre-training on large models and fine-tuning on downstream tasks, has provided new solutions for IR match tasks. In this study, we use the original BERT token in the embedding layer, improve the Sentence-BERT model structure in the model layer by introducing the SimCSE and K-Nearest Neighbors method, and use the cosent loss function in the optimization phase to optimize the target output. Our experimental results show that our model outperforms other competing models on both public and self-built datasets through comparative experiments and ablation implementations. This study explores and validates the feasibility and efficiency of pre-training techniques for semantic retrieval of Chinese scientific datasets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。