arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.00092cs.CLcs.AI
收藏

面向专利概念生成的大型语言模型

Large Language Model for Patent Concept Generation

City University of Hong Kong(香港城市大学)

Runtao Ren, Jian Ma, Jianxi Luo

AI总结:

本文提出知识微调框架并构建PatentGPT,通过知识注入预训练、领域监督微调和RLHF提升LLM在专利概念生成中的专业能力,显著超越现有模型。

摘要:

在传统创新实践中,概念生成与知识产权(IP)生成往往是迭代整合的。这两个过程都需要对先进技术领域知识有深入的理解。现有的大型语言模型(LLMs)虽然拥有海量的预训练知识,但由于缺乏生成所需的专业知识,在创新概念生成方面往往表现不足。为了弥合这一关键差距,我们提出了一种新颖的知识微调(KFT)框架,赋予基于LLM的人工智能自主挖掘、理解和应用领域特定知识与概念进行发明生成的能力,即同时进行概念生成和专利生成。我们提出的PatentGPT整合了知识注入预训练(KPT)、领域特定监督微调(SFT)和基于人类反馈的强化学习(RLHF)。广泛评估表明,PatentGPT在专利相关基准测试上显著优于最先进的模型。我们的方法不仅为数据驱动的创新提供了新的见解,也为在技术背景下微调LLMs用于应用开辟了新路径。我们还讨论了未来AI生成发明在管理和政策层面的影响。

英文摘要:

In traditional innovation practices, concept and IP generation are often iteratively integrated. Both processes demand an intricate understanding of advanced technical domain knowledge. Existing large language models (LLMs), while possessing massive pre-trained knowledge, often fall short in the innovative concept generation due to a lack of specialized knowledge necessary for the generation. To bridge this critical gap, we propose a novel knowledge finetuning (KFT) framework to endow LLM-based AI with the ability to autonomously mine, understand, and apply domain-specific knowledge and concepts for invention generation, i.e., concept and patent generation together. Our proposed PatentGPT integrates knowledge injection pre-training (KPT), domain-specific supervised finetuning (SFT), and reinforcement learning from human feedback (RLHF). Extensive evaluation shows that PatentGPT significantly outperforms the state-of-the-art models on patent-related benchmark tests. Our method not only provides new insights into data-driven innovation but also paves a new path to fine-tune LLMs for applications in the context of technology. We also discuss the managerial and policy implications of AI-generating inventions in the future.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑