有效的博客页面提取器更好的UGC搜索
Effective Blog Pages Extractor for Better UGC Accessing
摘要：
博客正在成为一种越来越流行的信息发布媒体。除了主要内容外，现在的博客页面中大多还包含广告等噪声信息。删除这些无关元素可以改善用户体验，也可以使内容更好地适应手机等各种设备。虽然基于模板的提取器是高度准确的，但它们可能会招致昂贵的成本，因为需要开发大量的模板，并且一旦模板被更新，它们就会失败。为了解决这些问题，我们提出了一种新颖的模板无关的内容提取器的博客页面。首先，我们将博客页面转换为DOM树，其中包括页面中的标题和主体块在内的所有元素都对应于子树。然后分别为标题块和正文块构造子树候选集，并提取子树中元素的空间和内容特征。使用这些特征训练标题和主体块的SVM分类器。最后，分类器被用来提取博客页面的主要内容。我们在从9个风格和模板明显不同的博客网站上抓取的2，250个博客页面上测试我们的提取器。实验结果验证了我们的提取器的有效性。
英文摘要：
Blog is becoming an increasingly popular media for information publishing. Besides the main content, most of blog pages nowadays also contain noisy information such as advertisements etc. Removing these unrelated elements can improves user experience, but also can better adapt the content to various devices such as mobile phones. Though template-based extractors are highly accurate, they may incur expensive cost in that a large number of template need to be developed and they will fail once the template is updated. To address these issues, we present a novel template-independent content extractor for blog pages. First, we convert a blog page into a DOM-Tree, where all elements including the title and body blocks in a page correspond to subtrees. Then we construct subtree candidate set for the title and the body blocks respectively, and extract both spatial and content features for elements contained in the subtree. SVM classifiers for the title and the body blocks are trained using these features. Finally, the classifiers are used to extract the main content from blog pages. We test our extractor on 2,250 blog pages crawled from nine blog sites with obviously different styles and templates. Experimental results verify the effectiveness of our extractor.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
