一种自动获取科技文献的搜索/抓取框架
A Search/Crawl Framework for Automatically Acquiring Scientific Documents
摘要：
尽管在搜索引擎功能、排名方法、技术和可编程API的可用性方面取得了进步，但当今的开放存取数字图书馆仍然依赖于基于抓取的方法来获取其底层文档集合。在本文中，我们提出了一种新的搜索驱动的框架，用于获取科学门户网站的文件。在我们的框架内，公开的研究论文标题和作者姓名被用作Web搜索引擎的查询。接着，使用精确的分类模块从搜索结果中识别研究论文和研究论文的来源。我们的实验不仅突出了我们的个人分类器的性能，而且我们的整体搜索/爬行框架的有效性。事实上，我们能够通过我们的全自动框架使用约7.6万个查询获得约66.5万份研究文档。这些多产的结果将Web搜索定位为抓取方法的有效替代方案，用于获取实际文档和种子URL以供将来抓取。
英文摘要：
Despite the advancements in search engine features, ranking methods, technologies, and the availability of programmable APIs, current-day open-access digital libraries still rely on crawl-based approaches for acquiring their underlying document collections. In this paper, we propose a novel search-driven framework for acquiring documents for scientific portals. Within our framework, publicly-available research paper titles and author names are used as queries to a Web search engine. Next, research papers and sources of research papers are identified from the search results using accurate classification modules. Our experiments highlight not only the performance of our individual classifiers but also the effectiveness of our overall Search/Crawl framework. Indeed, we were able to obtain approximately 0.665 million research documents through our fully-automated framework using about 0.076 million queries. These prolific results position Web search as an effective alternative to crawl methods for acquiring both the actual documents and seed URLs for future crawls.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
