arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.09443cs.CLcs.AI
收藏

多层级指令层级在大语言模型代理中的应用

Many-Tier Instruction Hierarchy in LLM Agents

Johns Hopkins University(约翰霍普金斯大学)

Jingyu Zhang, Tianjian Li, William Jurayj, Hongyuan Zhan, Benjamin Van Durme, Daniel Khashabi

AI总结:

本文提出Many-Tier Instruction Hierarchy (ManyIH) 以解决多源指令冲突,通过12层级的测试任务验证模型在复杂冲突场景下的表现,揭示了细粒度可扩展指令冲突解决的迫切需求。

摘要:

大语言模型代理接收来自多个来源的指令——系统信息、用户提示、工具输出、其他代理等——每种指令携带不同的信任等级和权威性。当这些指令冲突时,代理必须可靠地遵循最高特权指令以保持安全和有效。传统的指令层级(IH)假设一个固定的小特权层级集合(通常少于五个),由刚性的角色标签(如系统>用户)定义。这在现实世界代理设置中是不足的,因为冲突可能出现在更多来源和上下文中。本文提出Many-Tier Instruction Hierarchy(ManyIH),一种用于解决指令冲突的范式,允许任意多的特权层级。我们引入ManyIH-Bench,首个ManyIH基准测试,要求模型在多达12层级的冲突指令中导航,包含853个代理任务(427个编码任务和426个指令遵循任务)。ManyIH-Bench通过LLM开发并由人类验证约束,创建了覆盖46个现实世界代理的现实且困难的测试案例。我们的实验表明,即使当前前沿模型在指令冲突扩大时表现也较差(约40%的准确率)。本文强调了在代理设置中需要明确针对细粒度、可扩展的指令冲突解决方法的迫切需求。

英文摘要:

Large language model agents receive instructions from many sources-system messages, user prompts, tool outputs, other agents, and more-each carrying different levels of trust and authority. When these instructions conflict, agents must reliably follow the highest-privilege instruction to remain safe and effective. The dominant paradigm, instruction hierarchy (IH), assumes a fixed, small set of privilege levels (typically fewer than five) defined by rigid role labels (e.g., system > user). This is inadequate for real-world agentic settings, where conflicts can arise across far more sources and contexts. In this work, we propose Many-Tier Instruction Hierarchy (ManyIH), a paradigm for resolving instruction conflicts among instructions with arbitrarily many privilege levels. We introduce ManyIH-Bench, the first benchmark for ManyIH. ManyIH-Bench requires models to navigate up to 12 levels of conflicting instructions with varying privileges, comprising 853 agentic tasks (427 coding and 426 instruction-following). ManyIH-Bench composes constraints developed by LLMs and verified by humans to create realistic and difficult test cases spanning 46 real-world agents. Our experiments show that even the current frontier models perform poorly (~40% accuracy) when instruction conflict scales. This work underscores the urgent need for methods that explicitly target fine-grained, scalable instruction conflict resolution in agentic settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑