arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.00007cs.IRcs.CL
收藏

PyTorch-IE:面向信息抽取的快速且可复现的原型设计

PyTorch-IE: Fast and Reproducible Prototyping for Information Extraction

German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心)

Arne Binder, Leonhard Hennig, Christoph Alt

AI总结:

本文提出PyTorch-IE,一个基于深度学习的信息抽取框架,通过灵活数据模型与任务解耦设计,支持快速、可复现的IE模型开发,并集成主流库以助力研究社区。

摘要:

信息抽取(IE)的目标是从非结构化或半结构化文档中提取结构化表示。然而,由于需要整合多个子任务,开发IE模型十分复杂。此外,不同任务间的数据表示以及将数据集转换为特定于模型的输入也带来了进一步的挑战。为了简化研究人员的这一工作,我们引入了PyTorch-IE,这是一个基于深度学习的框架,专为快速、可复现和可重用地实现IE模型而设计。PyTorch-IE提供了一个灵活的数据模型,能够通过整合源自纯文本、半结构化文本甚至图像等多种数据类型的相互依赖的标注层,来创建复杂的数据结构。我们提出了任务模块,将数据表示和特定于模型的表示的关注点解耦,从而促进代码的灵活性和可重用性。PyTorch-IE还扩展支持了广泛使用的库,如用于训练的PyTorch-Lightning、用于数据集读取的HuggingFace datasets以及用于实验配置的Hydra。此外,还提供了辅助库和GitHub模板以便捷地设置新项目。通过确保功能性和通用性,PyTorch-IE为从事信息抽取的研究社区提供了重要支持。

英文摘要:

The objective of Information Extraction (IE) is to derive structured representations from unstructured or semi-structured documents. However, developing IE models is complex due to the need of integrating several subtasks. Additionally, representation of data among varied tasks and transforming datasets into task-specific model inputs presents further challenges. To streamline this undertaking for researchers, we introduce PyTorch-IE, a deep-learning-based framework uniquely designed to enable swift, reproducible, and reusable implementations of IE models. PyTorch-IE offers a flexible data model capable of creating complex data structures by integrating interdependent layers of annotations derived from various data types, like plain text or semi-structured text, and even images. We propose task modules to decouple the concerns of data representation and model-specific representations, thereby fostering greater flexibility and reusability of code. PyTorch-IE also extends support for widely used libraries such as PyTorch-Lightning for training, HuggingFace datasets for dataset reading, and Hydra for experiment configuration. Supplementary libraries and GitHub templates for the easy setup of new projects are also provided. By ensuring functionality and versatility, PyTorch-IE provides vital support to the research community engaged in Information Extraction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑