SkillSmith：学习组合参数化技能与文本知识

中文摘要 AI 辅助

由大语言模型（LLM）驱动的智能体系统，通常具备两种自主解决复杂问题的关键机制：从过往经验中合成基于文本的知识与流程，以及为重复出现的子目标构建参数化（权重空间）技能库。迄今为止，研究大多将这两者视为正交的任务：要么通过组合与反思组织文本知识，要么通过权重空间合并整合参数化技能。因此，针对目标性能提升的文本与模型权重的无缝集成，在很大程度上仍未被探索。本研究将模型权重视为大语言模型可自然推理的额外模态，以弥合这一模态差距。我们通过前缀调优实现参数化学习，并增强大语言模型，使其能够同时输入前缀权重与丰富的文本数据，这些数据捕捉了与目标能力的关联。我们的增强型大语言模型SkillSmith，会整合这些输入以执行指令引导的参数化合成，直接输出体现目标技能的新前缀权重。我们证明，我们的方法显著优于仅基于文本和仅基于权重空间的基线，解锁了单模态（仅文本或仅权重）适配无法实现的性能提升。

英文摘要

Agentic systems driven by large language models (LLMs) regularly feature two key mechanisms to autonomously solve complex problems: synthesizing text-based knowledge and procedures from past experiences and building parametric (weight-space) skill libraries for recurring sub-goals. To date, research has largely treated these as orthogonal pursuits: either organizing textual knowledge through composition and reflection, or consolidating parametric skills via weight-space merging. Consequently, the seamless integration of text and model weights for targeted performance improvements remains largely unexplored. This work bridges this modality gap by treating model weights as an additional modality that an LLM can natively reason over. We instantiate parametric learning via prefix-tuning and augment an LLM to ingest both prefix weights and rich textual data which capture relationships to a target capability. Our augmented LLM, which we call SkillSmith, synthesizes these inputs to perform instruction-steered parametric synthesis, directly outputting new prefix weights that manifest the target skill. We demonstrate that our approach significantly outperforms both text-only and weight-space-only baselines, unlocking performance gains that are out of reach for uni-modal (text-only or weight-only) adaptations.

发表机构

• Google（谷歌）

机构由 AI 辅助整理，请以论文原文为准。