arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1804.03056cs.CY
收藏

基于Web爬行的网络审查自动发现

Automated Discovery of Internet Censorship by Web Crawling

Alexander Darer, Oliver Farnan, Joss Wright

摘要:

对互联网的审查在世界各地都很普遍。随着对网络的访问变得越来越普遍,对该资源的过滤变得更加普遍。公民被拒绝访问的具体内容的透明度是不典型的。为了解决这个问题,许多个人和组织已经提出了许多用于维护URL过滤器列表的技术,这些技术旨在为公众和更广泛的审查研究社区提供关于审查的经验数据。我们提出了一种新的方法来发现过滤域在不同的国家。这种方法是完全自动化的,不需要人工干预。该系统使用网络爬行技术在过滤的站点之间遍历,并实现用于确定域是否被过滤的鲁棒方法。我们证明了该方法的有效性,通过运行实验来搜索过滤的内容在四个不同的审查制度。我们的研究结果表明,我们的表现优于当前最先进的技术,并且已经建立了比截至2018年1月最广泛使用的公共列表大一个数量级的域过滤器列表。此外,我们建立了一个数据集映射域之间的封锁内容的相互联系的性质,并展示了严格的网络审查的网络资源的性质。

英文摘要:

Censorship of the Internet is widespread around the world. As access to the web becomes increasingly ubiquitous, filtering of this resource becomes more pervasive. Transparency about specific content that citizens are denied access to is atypical. To counter this, numerous techniques for maintaining URL filter lists have been proposed by various individuals and organisations that aim to empirical data on censorship for benefit of the public and wider censorship research community. We present a new approach for discovering filtered domains in different countries. This method is fully automated and requires no human interaction. The system uses web crawling techniques to traverse between filtered sites and implements a robust method for determining if a domain is filtered. We demonstrate the effectiveness of the approach by running experiments to search for filtered content in four different censorship regimes. Our results show that we perform better than the current state of the art and have built domain filter lists an order of magnitude larger than the most widely available public lists as of Jan 2018. Further, we build a dataset mapping the interlinking nature of blocked content between domains and exhibit the tightly networked nature of censored web resources.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑