发表机构
School of Computer Science and Engineering, Southeast University; College of Software Engineering, Southeast University; Department of Computer Science and Engineering, The Chinese University of Hong Kong; Department of Computer Science, City University of Hong Kong(东南大学计算机科学与工程学院; 东南大学软件学院; 香港中文大学计算机科学与工程学系; 香港城市大学计算机科学系)
机构由 AI 辅助整理,请以论文原文为准。AI 中文总结
针对现有LLM智能体技能路由仅匹配任务的缺陷,提出SkillFeed渐进式检索-重排序框架,在自主构建的反事实基准上大幅提升了满足用户约束的技能检索准确率。
AI 中文摘要
可复用技能仓库的快速扩展使技能路由成为大语言模型(LLM)智能体的关键能力。现有方法将路由视为仅任务的语义匹配,但当具有不兼容约束的用户发出相同请求时,该假设混淆了任务相关性与技能适用性:仅任务路由器可能选择语义合理但不适合请求用户的技能。为揭示这种失败模式,我们将个性化技能路由建模为用户属性条件下的检索,其中相关性同时取决于任务和用户属性。我们首先引入用户属性反事实基准,在任务保持固定的同时,用户属性的变化会导致参考技能发生变化。我们进一步构建配对反事实监督,并提出SkillFeed,一种渐进式检索-重排序框架,该框架先建立任务-技能对齐,再学习用户属性条件下的判别。通过检索主体级证据并重排语义相似但用户属性冲突的候选,SkillFeed可识别同时满足任务要求和用户约束的技能。在SkillFeed-Bench上,SkillFeed达到75.1%的Top-1检索准确率,较对应的预训练路由基准提升23.1个百分点;在用户属性改变参考技能的查询中,加入用户属性条件带来35.1个百分点的增益。该对比表明,用户属性恰在改变技能适用性时最为关键。我们的网站可通过此http URL公开访问。
英文摘要
The rapid expansion of reusable skill repositories makes skill routing a critical capability for large language model (LLM) agents. Existing methods treat routing as task-only semantic matching. However, when users with incompatible constraints issue an identical request, this assumption conflates task relevance with skill suitability: a task-only router can select a semantically plausible skill that is unsuitable for the requesting user. To expose this failure mode, we formulate \textit{personalized skill routing} as profile-conditioned retrieval, in which relevance depends jointly on the task and the user profile. We first introduce a profile-counterfactual benchmark, in which the task is held fixed while changes in the user profile induce changes in the reference skill. We further construct paired counterfactual supervision and propose SkillFeed, a progressive retrieve-and-rerank framework that first establishes task--skill alignment and then learns profile-conditioned discrimination. By retrieving body-level evidence and reranking semantically similar but profile-conflicting candidates, SkillFeed identifies skills that satisfy both task requirements and user constraints. On SkillFeed-Bench, SkillFeed attains 75.1\% top-1 retrieval accuracy, a 23.1-point improvement over the corresponding pretrained routing baseline. Adding profile conditioning yields a 35.1-point gain on queries where user profile changes the reference skill. This contrast shows that user profiles are most consequential precisely when they change skill suitability. Our website is publicly available at http://www.aiskillfeed.com .