解读WWW的结构:事实与抽样偏差
Decoding the structure of the WWW: facts versus sampling biases
摘要:
了解万维网(WWW)巨大而复杂的拓扑结构是一个重大的科学和技术挑战。这已经解决了最近的特性,其代表图中的顶点和有向边被确定为网页和超链接,分别。在大规模爬行中收集的数据已经由几个小组进行了分析,从而得出了一个包含许多快速发展的网络典型的复杂属性的WWW的总体图。在本文中,我们报告了一个详细的统计分析的拓扑性质的四个不同的WWW图获得不同的爬虫。我们发现,尽管样本量非常大,但表征这些图的统计指标在数量上不同,在某些情况下在质量上也不同,这取决于分析的域和用于收集数据的抓取。这就引发了Web抓取的采样偏差和结构差异的存在问题,这些问题可能会导致不代表实际全局底层图的属性。为了提供一个更准确的表征的Web图形和识别可观的是明确区分相对于采样过程中,我们研究的度度相关函数的行为和互惠连接的统计。后者似乎包含了万维网图的相关关系,并携带了万维网的大部分拓扑信息。对这一数量的分析也与网络的可导航性和可搜索性有关。
英文摘要:
The understanding of the immense and intricate topological structure of the World Wide Web (WWW) is a major scientific and technological challenge. This has been tackled recently by characterizing the properties of its representative graphs in which vertices and directed edges are identified with web-pages and hyperlinks, respectively. Data gathered in large scale crawls have been analyzed by several groups resulting in a general picture of the WWW that encompasses many of the complex properties typical of rapidly evolving networks. In this paper, we report a detailed statistical analysis of the topological properties of four different WWW graphs obtained with different crawlers. We find that, despite the very large size of the samples, the statistical measures characterizing these graphs differ quantitatively, and in some cases qualitatively, depending on the domain analyzed and the crawl used for gathering the data. This spurs the issue of the presence of sampling biases and structural differences of Web crawls that might induce properties not representative of the actual global underlying graph. In order to provide a more accurate characterization of the Web graph and identify observables which are clearly discriminating with respect to the sampling process, we study the behavior of degree-degree correlation functions and the statistics of reciprocal connections. The latter appears to enclose the relevant correlations of the WWW graph and carry most of the topological information of theWeb. The analysis of this quantity is also of major interest in relation to the navigability and searchability of the Web.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。