FleetPilot:细粒度引导赋能真实场景下的大语言模型个性化
FleetPilot: Fine-Grained Steering Empowers Real-World LLM Personalization
AI总结:
针对现有LLM个性化方法的局限,提出FleetPilot细粒度引导框架,通过动态生成样本级干扰向量实现个性化适配,在多场景实验中表现优异且易部署。
摘要:
随着大语言模型(LLMs)的快速发展,对能使模型行为适配个体用户偏好的有效个性化技术的需求日益增长。个性化LLMs必须从有限、异构且不断演变的交互历史中推断用户依赖于查询的偏好。现有方法主要依赖上下文内个性化、参数适配或个性化奖励建模,但分别会在上下文管理、模型优化与部署,或多候选生成与评分中产生重复成本。当用户观测数据稀疏,且与每个查询相关的偏好证据随上下文和时间变化时,这些限制会变得尤为严格。为应对这些挑战,我们提出FleetPilot,这是一个细粒度且针对实例定制的引导框架,它从用户数据中动态生成样本级干扰向量,并将其注入模型的前向传播以实现个性化适配。我们的方法引入了:生成用户相关信号的匹配对比、通过捕获注意力层和MLP层的激活来捕捉细微信号的细粒度引导组件,以及将这些信号合成为上下文相关增强的查询条件聚合模块。在从短到长文本生成、网络函数调用等不同场景下的大量实验验证了FleetPilot的有效性和兼容性。结果表明,我们的方法持续带来显著的性能提升,且在偏好快速变化和数据稀疏的环境中表现出极强的优越性。此外,FleetPilot无需训练周期,仅产生少量的存储或推理延迟开销,这些特性使FleetPilot成为一种强大且可实际部署的方法。实现代码可在以下网址获取:this https URL
英文摘要:
With the rapid evolution of large language models (LLMs), there is an increasing demand for effective personalization techniques that can adapt model behavior to individual user preferences. Personalized LLMs must infer a user's query-dependent preferences from limited, heterogeneous, and evolving interaction histories. Existing approaches rely primarily on in-context personalization, parametric adaptation, or personalized reward modeling, but respectively incur recurring costs in context management, model optimization and deployment, or multi-candidate generation and scoring. These limitations become particularly restrictive when user observations are sparse and the preference evidence relevant to each query varies across contexts and over time. To address these challenges, we propose \modelname, a fine-grained and instance-tailored steering framework that dynamically generates sample-level interference vectors from user data and injects them into the model's forward pass for personalized adaptation. Our approach introduces: match contrast that produces user-relevant signals, fine-grained steering component that captures nuanced signals by hooking activations from attention and MLP layers, and query-conditioned aggregation module that synthesizes these signals into contextually relevant enhancements. Extensive experiments across diverse scenarios, including short-to-long text generation, and web function calling, validate the effectiveness and compatibility of \modelname. Results show that our method consistently brings significant performance gains, and show extreme superiority in fast-shifting and data-sparse environments. Additionally, \modelname rids us of training cycles and incurs light overhead of storage or inference latency. These make \modelname a strong and real-world deployable method. Implementation is available \footnote{https://anonymous.4open.science/r/FleetPilot}.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。