追踪自适应智能体的行为轨迹
Tracking the Behavioral Trajectories of Adapting Agents
AI总结:
提出一种通过文本嵌入空间中的方向定义智能体特质的方法,训练线性模型对技能文件差异进行评分,实现高准确率的行为特质分类与排序。
摘要:
文本文件,如技能文件、记忆文件和行为配置文件,在定义现代智能体的行为方式中起着核心作用。通过人类或智能体自身的编辑,这些文件可能随时间演变,直接引导智能体在未来交互中的行为。我们提出了一种方法和框架,通过将特质定义为文本嵌入模型嵌入空间中的方向来测量智能体的“特质”。我们在标记的“之前”与“之后”技能文件差异上训练线性模型以学习特质向量,然后通过将嵌入差异投影到该向量上对任意技能编辑进行评分。在68个标记的技能差异对上评估寻求敏感数据的特质倾向,我们的方法在留一法交叉验证下实现了91.2%的符号分类准确率和斯皮尔曼等级相关系数ρ=0.82。我们将这种特质评估构建到一个更广泛的智能体间协议中,使一个智能体能够通过可信中介评估另一个智能体的技能文件更新。
英文摘要:
Text files such as skill files, memory files, and behavioral configuration files play a central role in defining how modern agents act. Through edits by humans or the agents themselves, these files may evolve over time, directly steering the agent's behavior in future interactions. We present a methodology and framework for measuring agent $traits$ by defining traits as directions in the embedding space of a text embedding model. We train a linear model on labeled "before" versus "after" skill file diffs to learn a trait vector, then score arbitrary skill edits by projecting their embedding diffs onto this vector. Evaluated on 68 labeled skill diff pairs for the trait of propensity to seek sensitive data, our method achieves 91.2% sign classification accuracy and a Spearman rank correlation of $ρ= 0.82$ under leave-one-out cross-validation. We build this trait evaluation into a broader agent-to-agent protocol that enables one agent to evaluate another's skill file updates through a trusted intermediary.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。