生成向量搜索以提升多模态视觉-语言任务中的病理基础模型
Generative vector search to improve pathology foundation models across multimodal vision-language tasks
AI总结:
STHLM通过生成向量搜索方法提升多模态视觉-语言任务中病理基础模型的检索性能,实现10-30%的性能提升和10倍的维度压缩
摘要:
检索增强生成通过将输出与外部知识源相结合,提高了大语言模型的性能,减少了幻觉并解决了知识截止问题。然而,标准基于嵌入的检索无法捕捉多概念查询的复杂性,尤其是在生物医学领域,其中生物数据本质上是高维的。例如,组学数据集和临床报告同时表现出众多分子、细胞和生理特征。我们提出了随机潜在匹配(STHLM),一种生成向量搜索方法,通过从文本或图像输入中采样查询条件嵌入来增强检索性能。类似于链式思维推理使语言模型能够“更深入地”思考复杂问题,STHLM使检索系统能够“更广泛地”搜索通过迭代采样。STHLM在各种基准测试中均优于传统向量检索,包括科学文献、临床笔记和组织图像,通过测试时计算(用延迟换取准确性)将检索性能提高10-30%,同时使嵌入维度压缩达10倍。
英文摘要:
Retrieval-augmented generation improves large language models by grounding outputs in external knowledge sources, reducing hallucinations and addressing knowledge cutoffs. However, standard embedding-based retrieval fails to capture the complexity of multi-concept queries, particularly in domains like biomedicine, where biological data are inherently high-dimensional. For example,omics datasets, and clinical reports simultaneously exhibit numerous molecular, cellular, and physiological features. We present Stochastic Latent Matching (STHLM), a generative vector search method that samples query-conditioned embeddings from text or image inputs to enhance retrieval performance. Analogous to how Chain-of-Thought reasoning enables language models to "think longer" on complex problems, STHLM allows retrieval systems to "search wider" through iterative sampling. STHLM demonstrates critical improvements over classical vector retrieval across diverse benchmarks, including scientific literature, clinical notes, and tissue images, boosting retrieval performance by 10-30% through test-time compute (trading latency for accuracy), while enabling up to a 10-fold compression of embedding dimensions.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。