PatenTEB:面向专利文本嵌入的综合基准与模型家族
PatenTEB: A Comprehensive Benchmark and Model Family for Patent Text Embedding
AI总结:
针对专利文本嵌入缺乏专门基准的问题,提出PatenTEB基准(15任务、206万样本)及patembed模型家族,通过多任务训练和领域预训练实现外部泛化性能领先。
摘要:
专利文本嵌入能够支持现有技术检索、技术全景分析和专利分析,然而现有基准未能充分捕捉专利领域的特有挑战。我们提出了PatenTEB,一个包含15项任务的综合基准,涵盖检索、分类、释义和聚类,共包含206万个样本。PatenTEB采用领域分层划分、领域特定的困难负样本挖掘,并系统覆盖了通用嵌入基准中缺失的非对称片段到文档匹配场景。我们通过多任务训练开发了patembed模型家族,参数量从6700万到3.44亿不等,上下文长度可达4096个token。外部验证显示出强大的泛化能力:patembed-base在MTEB BigPatentClustering.v2上达到了最先进水平(V-measure为0.494,优于此前最佳的0.445),而patembed-large在DAPFAM上达到了0.377的NDCG@100。系统的消融实验表明,尽管多任务训练带来少量基准性能代价,但它提升了外部泛化能力,并且领域预训练初始化在各个任务族中提供了一致的优势。所有资源将在https://github.com/iliass-y/patenteb上提供。关键词:专利检索、句子嵌入、多任务学习、非对称检索、基准评估、对比学习。
英文摘要:
Patent text embeddings enable prior art search, technology landscaping, and patent analysis, yet existing benchmarks inadequately capture patent-specific challenges. We introduce PatenTEB, a comprehensive benchmark comprising 15 tasks across retrieval, classification, paraphrase, and clustering, with 2.06 million examples. PatenTEB employs domain-stratified splits, domain specific hard negative mining, and systematic coverage of asymmetric fragment-to-document matching scenarios absent from general embedding benchmarks. We develop the patembed model family through multi-task training, spanning 67M to 344M parameters with context lengths up to 4096 tokens. External validation shows strong generalization: patembed-base achieves state-of-the-art on MTEB BigPatentClustering.v2 (0.494 V-measure vs. 0.445 previous best), while patembed-large achieves 0.377 NDCG@100 on DAPFAM. Systematic ablations reveal that multi-task training improves external generalization despite minor benchmark costs, and that domain-pretrained initialization provides consistent advantages across task families. All resources will be made available at https://github.com/iliass-y/patenteb. Keywords: patent retrieval, sentence embeddings, multi-task learning, asymmetric retrieval, benchmark evaluation, contrastive learning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。