arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.12522cs.CLcs.AI
收藏

Struct-X:利用结构化数据增强大语言模型推理能力

Struct-X: Enhancing Large Language Models Reasoning with Structured Data

Shanghai University of Engineering Science(上海工程技术大学);INF Technology(Shanghai) Co., Ltd.(INF科技(上海)有限公司);Fudan University(复旦大学)

Xiaoyu Tan, Haoyu Wang, Xihe Qiu, Yuan Cheng, Yinghui Xu, Wei Chu, Yuan Qi

AI总结:

提出Struct-X框架,通过读取-建模-填充-反思-推理五阶段利用结构化数据,减少令牌冗余,增强LLM推理,在知识图谱问答和长文档阅读任务上显著提升性能。

摘要:

结构化数据富含逻辑和关系信息,有潜力增强大语言模型(LLM)的推理能力。然而,由于其可能用过多令牌和不相关上下文信息淹没LLM,其整合面临挑战。为解决此问题,我们提出Struct-X,一种新颖框架,通过五个关键阶段运作:“读取-建模-填充-反思-推理”,高效使LLM利用结构化数据。它首先通过图嵌入将结构化数据编码到拓扑空间,随后用知识检索模块填补缺失实体信息,并通过自监督模块过滤不相关令牌。最后阶段涉及用选定令牌构建拓扑网络,进一步减少总令牌长度,以实现更有效的LLM推理。此外,Struct-X包含一个辅助模块,训练生成提示,帮助LLM分析结构化数据。在包括知识图谱问答任务和长文档阅读理解任务在内的基准上的广泛实验表明,Struct-X显著改善LLM推理,证明结构化数据增强在复杂输入上下文中改善LLM推理的有效性。

英文摘要:

Structured data, rich in logical and relational information, has the potential to enhance the reasoning abilities of large language models (LLMs). Still, its integration poses a challenge due to the risk of overwhelming LLMs with excessive tokens and irrelevant context information. To address this, we propose Struct-X, a novel framework that operates through five key phases: ``read-model-fill-reflect-reason'' efficiently enabling LLMs to utilize structured data. It begins by encoding structured data into a topological space using graph embeddings, followed by filling in missing entity information with knowledge retrieval modules, and filtering out irrelevant tokens via a self-supervised module. The final phase involves constructing a topological network with selected tokens to further reduce the total token length for more effective LLM inference. Additionally, Struct-X includes an Auxiliary Module trained to generate prompts, aiding LLMs in analyzing structured data. Extensive experiments on benchmarks, including the knowledge graph question-answer task and the long document reading comprehension task, show that Struct-X notably improves LLM reasoning, demonstrating the effectiveness of structured data augmentation in improving LLM inference with complex input context.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑