从分析到综合：个性化大语言模型智能体中的隐式行为对齐基准测试

中文摘要 AI 辅助

大语言模型已催生能力日益强大的自主智能体，但个性化对让这类智能体具备实际实用性仍至关重要。近期的基准测试已开始评估智能体的个性化能力，但它们大多依赖静态偏好快照、固定交互日志或针对预定义用户档案的问答。这类设计无法捕捉不断变化的用户偏好的复杂性，也忽视了偏好条件下的任务执行——我们将这种差异称为“知识到行动的鸿沟”。为应对这一挑战，我们推出IBA-Bench，这是一个基于包含噪声、隐式线索和时间不一致性的纵向交互历史构建的隐式行为对齐基准。与现有研究不同，IBA-Bench评估智能体能否在执行任务时满足从历史交互中推断出的隐式用户约束。我们进一步提出IBA-Agent，这一智能体框架通过广泛检索和轨迹级对齐来协调冲突的优先级。在IBA-Bench上的实验结果表明，有效的个性化仍是当前最先进的大语言模型智能体面临的重大挑战，而所提出的IBA-Agent在九个应用领域的复杂场景中显著提升了行为对齐效果。

英文摘要

Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful. Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles. Such designs fail to capture the complexity of evolving user preferences and neglect preference-conditioned task execution-a discrepancy we term as the knowledge-to-action gap. To address this challenge, we introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories that contain noise, implicit cues, and temporal inconsistencies. Unlike prior work, IBA-Bench evaluates whether an agent can execute tasks while satisfying implicit user constraints inferred from historical interactions. We further propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment. Experiment results on IBA-Bench show that effective personalization remains a significant challenge for state-of-the-art LLM agents, and the proposed IBA-Agent substantially improves behavioral alignment in complex scenarios across nine application domains.

发表机构

• National University of Singapore（新加坡国立大学）
• University of Toronto（多伦多大学）
• University of Minnesota Twin Cities（明尼苏达大学双城分校）
• Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））
• University of Oxford（牛津大学）

机构由 AI 辅助整理，请以论文原文为准。