Web上联合搜索的资源选择
Resource Selection for Federated Search on the Web
摘要：
长期以来，一直没有一个反映真实网络环境的联邦搜索公开数据集，这使得研究人员很难测试他们的联邦搜索算法在网络环境中的有效性。我们提出了几个实验和分析资源选择在网络上使用最近发布的测试集合，包含从100多个真正的搜索引擎的结果，从大型通用网络搜索引擎，如谷歌，必应和雅虎小型特定领域的引擎。首先，我们尝试使用基于查询的采样来估计网络上不合作的搜索引擎的大小，并提出了一种使用CNOW Web09数据集的新方法。我们发现规模估计是非常有效的资源选择。第二，我们表明，一个优化的联邦搜索系统的基础上较小的网络搜索引擎可以替代系统使用大型网络搜索引擎。第三，我们提供了几个流行的资源选择方法的实证比较，发现这些方法是不容易适合在网络上的资源选择。挑战包括稀疏的资源描述和集合的大小极度倾斜。
英文摘要：
A publicly available dataset for federated search reflecting a real web environment has long been absent, making it difficult for researchers to test the validity of their federated search algorithms for the web setting. We present several experiments and analyses on resource selection on the web using a recently released test collection containing the results from more than a hundred real search engines, ranging from large general web search engines such as Google, Bing and Yahoo to small domain-specific engines. First, we experiment with estimating the size of uncooperative search engines on the web using query based sampling and propose a new method using the ClueWeb09 dataset. We find the size estimates to be highly effective in resource selection. Second, we show that an optimized federated search system based on smaller web search engines can be an alternative to a system using large web search engines. Third, we provide an empirical comparison of several popular resource selection methods and find that these methods are not readily suitable for resource selection on the web. Challenges include the sparse resource descriptions and extremely skewed sizes of collections.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
