TEXT2DB：基于大语言模型代理的集成感知信息抽取
TEXT2DB: Integration-Aware Information Extraction with Large Language Model Agents
AI总结：
提出TEXT2DB任务以解决IE输出与下游应用需求不匹配问题，并构建新基准与OPAL代理框架，通过观察、规划、分析组件协同生成代码计划并调用IE模型，成功实现跨数据库模式的自适应信息集成。
摘要：
信息抽取（IE）任务旨在从文本中提取结构化知识。然而，由于IE本体与下游应用需求之间的不匹配，利用IE输出通常并不简单。我们提出了一种新的IE任务表述TEXT2DB，强调将IE输出与目标数据库（或知识库）进行集成。给定用户指令、文档集和数据库，我们的任务要求模型利用文档集中的值更新数据库，以满足用户指令。该任务要求模型理解用户指令以确定抽取内容，并动态适应给定的DB/KB模式以决定如何抽取。为了评估这一新任务，我们引入了一个新基准，包含数据填充、行填充和列添加等常见需求。此外，我们提出了一个LLM代理框架OPAL（Observe-Plan-Analyze LLM），包括与数据库交互的Observer组件、生成基于代码的计划并调用IE模型的Planner组件，以及在执行前提供代码质量反馈的Analyzer组件。实验表明，OPAL能够通过生成不同的代码计划并调用所需的IE模型，成功适应多样化的数据库模式。我们还强调了处理具有复杂依赖关系的大型数据库和抽取幻觉等困难情况，这些值得进一步研究。源代码：https://github.com/yzjiao/Text2DB
英文摘要：
The task of information extraction (IE) is to extract structured knowledge from text. However, it is often not straightforward to utilize IE output due to the mismatch between the IE ontology and the downstream application needs. We propose a new formulation of IE TEXT2DB that emphasizes the integration of IE output and the target database (or knowledge base). Given a user instruction, a document set, and a database, our task requires the model to update the database with values from the document set to satisfy the user instruction. This task requires understanding user instructions for what to extract and adapting to the given DB/KB schema for how to extract on the fly. To evaluate this new task, we introduce a new benchmark featuring common demands such as data infilling, row population, and column addition. In addition, we propose an LLM agent framework OPAL (Observe-PlanAnalyze LLM) which includes an Observer component that interacts with the database, the Planner component that generates a code-based plan with calls to IE models, and the Analyzer component that provides feedback regarding code quality before execution. Experiments show that OPAL can successfully adapt to diverse database schemas by generating different code plans and calling the required IE models. We also highlight difficult cases such as dealing with large databases with complex dependencies and extraction hallucination, which we believe deserve further investigation. Source code: https://github.com/yzjiao/Text2DB
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
