arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.15074cs.CVcs.AI
收藏

PatentLMM:用于生成专利附图描述的大型多模态模型

PatentLMM: Large Multimodal Model for Generating Descriptions for Patent Figures

Indian Institute of Technology Jodhpur(焦特布尔印度理工学院);Microsoft(微软公司)

Shreya Shukla, Nakul Sharma, Manish Gupta, Anand Mishra

AI总结:

针对专利附图描述自动化任务,提出大规模数据集PatentDesc-355K及多模态模型PatentLMM,通过专用视觉编码器与领域微调LLaMA生成高质量描述,显著优于现成模型。

摘要:

在专利文档中撰写全面准确的技术图纸描述对于有效知识共享以及实现知识产权的复制和保护至关重要。然而,该任务的自动化在很大程度上被研究界所忽视。为此,我们引入了PatentDesc-355K,这是一个新的大规模数据集,包含约355K张专利附图及其从超过60K份美国专利文档中提取的简短和详细文本描述。此外,我们提出了PatentLMM——一种专门定制用于生成高质量专利附图描述的新型多模态大型语言模型。我们提出的PatentLMM包含两个关键组件:(i)PatentMME,一种专门用于捕捉专利附图独特结构元素的多模态视觉编码器,以及(ii)PatentLLaMA,在大量专利集合上微调的LLaMA领域适应版本。大量实验表明,训练专门为专利附图设计的视觉编码器显著提升了性能,与微调类似规模的现成多模态模型相比,能生成连贯的描述。PatentDesc-355K和PatentLMM为自动化理解专利附图铺平了道路,实现了高效的知识共享和更快的专利文档起草。我们公开了代码和数据。

英文摘要:

Writing comprehensive and accurate descriptions of technical drawings in patent documents is crucial to effective knowledge sharing and enabling the replication and protection of intellectual property. However, automation of this task has been largely overlooked by the research community. To this end, we introduce PatentDesc-355K, a novel large-scale dataset containing ~355K patent figures along with their brief and detailed textual descriptions extracted from more than 60K US patent documents. In addition, we propose PatentLMM - a novel multimodal large language model specifically tailored to generate high-quality descriptions of patent figures. Our proposed PatentLMM comprises two key components: (i) PatentMME, a specialized multimodal vision encoder that captures the unique structural elements of patent figures, and (ii) PatentLLaMA, a domain-adapted version of LLaMA fine-tuned on a large collection of patents. Extensive experiments demonstrate that training a vision encoder specifically designed for patent figures significantly boosts the performance, generating coherent descriptions compared to fine-tuning similar-sized off-the-shelf multimodal models. PatentDesc-355K and PatentLMM pave the way for automating the understanding of patent figures, enabling efficient knowledge sharing and faster drafting of patent documents. We make the code and data publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑