PORTS：用于大语言模型工具选择的偏好优化检索器
PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models
AI总结：
研究针对大语言模型工具选择中现有检索器与LLMs不一致问题，提出PORTS方法，利用受困惑度启发的偏好信号，通过优化相关性及施加对比语义损失微调检索器，经多实验验证其通用性及提高工具选择准确性的能力，且计算需求低便于推广。
摘要：
将外部工具与大语言模型（LLMs）集成已成为完成复杂任务的一种有前景的范式。由于LLMs难以有效管理大型工具集，研究人员开始探索基于检索的方法来预先选择最相关的选项。然而，现有检索器因其单独的训练过程，常与调用工具的LLMs不一致。本文提出PORTS，一种用于工具选择训练检索器的新颖优势比偏好优化方法。利用来自冻结LLM的受困惑度启发的偏好信号，通过优化选择概率与下游性能之间的相关性，同时在文档字符串之间强制施加对比语义损失，对检索器进行微调以找到有用工具。通过在六个数据集、两个编码器模型和三个具有不同先验知识的LLMs上进行广泛实验，证明了PORTS的通用性及其显著提高工具选择准确性的能力。其计算需求低，对齐过程便于推广到新查询和工具，对不断发展的工具集的实际应用很有价值。
英文摘要：
Integrating external tools with Large Language Models (LLMs) has emerged as a promising paradigm for accomplishing complex tasks. Since LLMs still struggle to effectively manage large tool collections, researchers have begun exploring retrieval-based methods to pre-select the most relevant options, addressing input length and latency constraints. However, existing retrievers are often misaligned with tool-calling LLMs due to their separate training processes. This paper presents PORTS, a novel odds ratio preference optimization method for training retrievers aimed at tool selection. Using a perplexity-inspired preference signal from a frozen LLM, our approach fine-tunes a retriever to find helpful tools by optimizing the correlation between the selection probabilities and the downstream performances while jointly enforcing a contrastive semantic loss between documentation strings. The versatility of PORTS and its ability to significantly improve tool selection accuracy are demonstrated through extensive experiments on six datasets, two encoder models, and three LLMs with diverse prior knowledge. With low computational demands, our alignment process facilitates generalization to new queries and tools, proving valuable for practical applications with evolving toolsets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
