arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2108.01454cs.IR
收藏

Inscriptis --一个基于Python的HTML到文本转换库,为从Web提取知识而优化

Inscriptis -- A Python-based HTML to text conversion library optimized for knowledge extraction from the Web

Albert Weichselbraun

摘要:

Inscriptis提供了一个库、命令行客户端和Web服务,用于将HTML转换为纯文本。它的发展是由需要获得准确的文本表示的知识提取任务,保留文本的空间对齐,而不利用重量级的,基于浏览器的解决方案,如Selenium。与相关的软件包相比,Inscriptis(i)提供HTML的布局感知转换,其更接近于从标准Web浏览器获得的渲染;以及(ii)支持注释规则,即,用户提供的映射,允许基于HTML标记和属性中编码的结构和语义信息注释提取的文本。这些独特的功能确保了下游的知识提取组件可以对准确的文本表示进行操作,甚至可以使用原始HTML文档的语义和结构信息。

英文摘要:

Inscriptis provides a library, command line client and Web service for converting HTML to plain text. Its development has been triggered by the need to obtain accurate text representations for knowledge extraction tasks that preserve the spatial alignment of text without drawing upon heavyweight, browser-based solutions such as Selenium. In contrast to related software packages, Inscriptis (i) provides a layout-aware conversion of HTML that more closely resembles the rendering obtained from standard Web browsers; and (ii) supports annotation rules, i.e., user-provided mappings that allow for annotating the extracted text based on structural and semantic information encoded in HTML tags and attributes. These unique features ensure that downstream knowledge extraction components can operate on accurate text representations, and may even use information on the semantics and structure of the original HTML document.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑