LasUIE：用潜在自适应结构感知生成语言模型统一信息抽取
LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model
AI总结：
本文提出结构感知生成语言模型LasUIE，通过异构结构归纳、结构广播和任务导向微调，将句法知识充分融入统一信息抽取，在12个基准上显著超越基线并缓解长距离依赖与边界识别难题。
摘要：
最新研究揭示，用单一生成语言模型（GLM）统一建模所有典型信息抽取任务（UIE）具有巨大潜力，其中各种信息抽取预测被统一为GLM下的线性化层次表达式。句法结构信息作为信息抽取领域广泛使用的一类有效特征，也应当对UIE有益。在本工作中，我们提出一种新颖的结构感知GLM，充分释放句法知识对UIE的作用。我们探索了一种异构结构归纳器，通过对现有GLM进行后训练，以无监督方式诱导丰富的异构结构表示。特别地，我们设计了一种结构广播器，将各种潜在树压缩为显式高阶森林，有助于在解码过程中引导更好的生成。最后，我们引入一种面向任务的结构微调机制，进一步调整所学结构以最大程度契合最终任务的需求。在7个任务、12个信息抽取基准上，我们的系统相比基线UIE系统取得了显著提升。进一步的深入分析表明，我们的GLM学习到了丰富的任务自适应结构偏置，极大地解决了UIE的关键难题——长距离依赖问题和边界识别问题。源代码已在 https://github.com/ChocoWu/LasUIE 开源。
英文摘要：
Universally modeling all typical information extraction tasks (UIE) with one generative language model (GLM) has revealed great potential by the latest study, where various IE predictions are unified into a linearized hierarchical expression under a GLM. Syntactic structure information, a type of effective feature which has been extensively utilized in IE community, should also be beneficial to UIE. In this work, we propose a novel structure-aware GLM, fully unleashing the power of syntactic knowledge for UIE. A heterogeneous structure inductor is explored to unsupervisedly induce rich heterogeneous structural representations by post-training an existing GLM. In particular, a structural broadcaster is devised to compact various latent trees into explicit high-order forests, helping to guide a better generation during decoding. We finally introduce a task-oriented structure fine-tuning mechanism, further adjusting the learned structures to most coincide with the end-task's need. Over 12 IE benchmarks across 7 tasks our system shows significant improvements over the baseline UIE system. Further in-depth analyses show that our GLM learns rich task-adaptive structural bias that greatly resolves the UIE crux, the long-range dependence issue and boundary identifying. Source codes are open at https://github.com/ChocoWu/LasUIE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
