超越扩展:使用领域特定细粒度权利要求依赖图预测专利审批
Beyond Scaling: Predicting Patent Approval with Domain-specific Fine-grained Claim Dependency Graph
AI总结:
本文针对专利审批预测任务,提出了一种新颖的细粒度权利要求依赖图(FLAN Graph),通过捕捉专利文本片段间的内在依赖关系并结合低成本图模型,显著优于单纯扩大LLM模型规模的方法。
摘要:
由于大型语言模型(LLMs)的成功,模型扩展正成为许多语言任务的默认选择。然而,在简单的定制化方法表现优异的特定场景中,它可能会显得不足。在本文中,我们深入探讨了专利审批预测任务,并揭示了简单的领域特定图方法利用专利数据内的内在依赖关系,其表现优于扩大模型规模。具体而言,我们首先通过使用各种规模的开源LLMs扩大其骨干模型来扩展基于嵌入的最先进(SOTA)方法,然后探索基于提示的方法以利用专有LLMs的潜力,但发现最佳结果接近于随机猜测,这凸显了模型扩展的无效性。因此,我们通过对专利数据的细致分析,提出了一种新颖的细粒度权利要求依赖(FLAN)图,捕捉专利文本各片段间的内在依赖关系。由于它具有模型无关性,我们将低成本的图模型应用于我们的FLAN图,以获取用于审批预测的表示。大量实验和详细分析证明,通过各种图模型结合FLAN图,始终显著优于所有LLM基线。我们希望本文中的观察和分析能为这一具有挑战性的任务带来更多关注,并促使对LLMs局限性的进一步研究。我们的源代码和数据集可从http://github.com/ShangDataLab/FLAN-Graph获取。
英文摘要:
Model scaling is becoming the default choice for many language tasks due to the success of large language models (LLMs). However, it can fall short in specific scenarios where simple customized methods excel. In this paper, we delve into the patent approval pre-diction task and unveil that simple domain-specific graph methods outperform enlarging the model, using the intrinsic dependencies within the patent data. Specifically, we first extend the embedding-based state-of-the-art (SOTA) by scaling up its backbone model with various sizes of open-source LLMs, then explore prompt-based methods to harness proprietary LLMs' potential, but find the best results close to random guessing, underlining the ineffectiveness of model scaling-up. Hence, we propose a novel Fine-grained cLAim depeNdency (FLAN) Graph through meticulous patent data analyses, capturing the inherent dependencies across segments of the patent text. As it is model-agnostic, we apply cost-effective graph models to our FLAN Graph to obtain representations for approval prediction. Extensive experiments and detailed analyses prove that incorporating FLAN Graph via various graph models consistently outperforms all LLM baselines significantly. We hope that our observations and analyses in this paper can bring more attention to this challenging task and prompt further research into the limitations of LLMs. Our source code and dataset can be obtained from http://github.com/ShangDataLab/FLAN-Graph.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。