arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2412.09280cs.CL
收藏

基于知识密集型程序生成器学习求解领域特定计算问题

Learning to Solve Domain-Specific Calculation Problems with Knowledge-Intensive Programs Generator

College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);Alibaba Group(阿里巴巴集团)

Chengyuan Liu, Shihang Wang, Lizhi Qing, Jun Lin, Ji Zhang, Fei Wu, Kun Kuang

AI总结:

针对大语言模型在涉及复杂领域规则与知识的计算问题上的不足,提出知识密集型程序生成器KIPG流水线,通过提取变量、生成程序、迭代偏好对齐求解问题,在法律领域验证有效且可跨领域适配。

摘要:

领域大语言模型(LLMs)是在通用大语言模型基础上为领域特定任务开发的,但部分领域特定任务仍需专业知识来支撑其专业性。本文针对知识密集型计算问题展开研究,发现当数学问题涉及复杂的领域特定规则与知识文档、而非简单的术语表述时,对大语言模型而言颇具挑战性。为此,我们提出一种名为KIPG(知识密集型程序生成器)的流水线,以更高效地求解领域特定计算问题,它能依据领域特定文档生成知识密集型程序。针对每个查询,系统先提取关键变量,再借助程序计算依赖领域知识的结果。通过迭代偏好对齐,代码生成器可学习提升与领域知识的逻辑一致性。我们以法律领域为例开展实验,验证了该流水线的有效性,并对各模块进行了深入分析,同时发现该代码生成器无需在新知识上训练,即可适配其他领域。

英文摘要:

Domain Large Language Models (LLMs) are developed for domain-specific tasks based on general LLMs. But it still requires professional knowledge to facilitate the expertise for some domain-specific tasks. In this paper, we investigate into knowledge-intensive calculation problems. We find that the math problems to be challenging for LLMs, when involving complex domain-specific rules and knowledge documents, rather than simple formulations of terminologies. Therefore, we propose a pipeline to solve the domain-specific calculation problems with Knowledge-Intensive Programs Generator more effectively, named as KIPG. It generates knowledge-intensive programs according to the domain-specific documents. For each query, key variables are extracted, then outcomes which are dependent on domain knowledge are calculated with the programs. By iterative preference alignment, the code generator learns to improve the logic consistency with the domain knowledge. Taking legal domain as an example, we have conducted experiments to prove the effectiveness of our pipeline, and extensive analysis on the modules. We also find that the code generator is also adaptable to other domains, without training on the new knowledge.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑