arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.16897cs.CL
收藏

InstructPatentGPT:训练专利语言模型以遵循人类反馈指令

InstructPatentGPT: Training patent language models to follow instructions with human feedback

National Yang Ming Chiao Tung University(国立阳明交通大学);School of Law(法学院)

Jieh-Sheng Lee

AI总结:

本研究将专利审查概念化为基于人类反馈的强化学习系统,通过三阶段训练使语言模型生成更易授权的专利权利要求,并在单GPU上验证了概念可行性。

摘要:

在这项研究中,专利审查被概念化为一个基于人类反馈的强化学习系统。该系统的目标是提高语言模型生成具有更高授权概率的专利权利要求的可能性。为了展示语言模型的可控性,系统从具有不同奖励的已授权专利和授权前申请中进行学习。“已授权”和“授权前”的状态被视为隐式标记的人类反馈。此外,针对专利起草,本研究中的实验证明了模型通过调整权利要求长度和包含限制性术语以缩小权利要求范围的学习能力。作为概念验证,实验仅关注独立权利要求,训练数据源自专门为人工智能定制的专利数据集。尽管专利审查中可用的人类反馈有限,且生成的专利文本质量有待提高,但遵循三阶段基于人类反馈的强化学习的实验表明,生成式语言模型能够反映专利审查中的人类反馈或意图。为了增强语言模型的可用性,本研究的实现采用了能够在单个消费级GPU上执行的现代技术。这一展示的概念验证降低了硬件要求,未来随着专利审查中出现更多可供广泛使用的人类反馈,无论是在专利局内部还是在公共领域,都将证明其价值。

英文摘要:

In this research, patent prosecution is conceptualized as a system of reinforcement learning from human feedback. The objective of the system is to increase the likelihood for a language model to generate patent claims that have a higher chance of being granted. To showcase the controllability of the language model, the system learns from granted patents and pre-grant applications with different rewards. The status of "granted" and "pre-grant" are perceived as labeled human feedback implicitly. In addition, specific to patent drafting, the experiments in this research demonstrate the model's capability to learn from adjusting claim length and inclusion of limiting terms for narrowing claim scope. As proof of concept, the experiments focus on claim ones only and the training data originates from a patent dataset tailored specifically for artificial intelligence. Although the available human feedback in patent prosecution are limited and the quality of generated patent text requires improvement, the experiments following the 3-stage reinforcement learning from human feedback have demonstrated that generative language models are capable of reflecting the human feedback or intent in patent prosecution. To enhance the usability of language models, the implementation in this research utilizes modern techniques that enable execution on a single consumer-grade GPU. The demonstrated proof of concept, which reduces hardware requirements, will prove valuable in the future as more human feedback in patent prosecution become available for broader use, either within patent offices or in the public domain.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑