arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~

被叙事诱惑:评估半开放文本沙盒中的规则遵守情况

Assessing Rule Adherence of LLM Adjudicators in Call of Cthulhu TRPG

Weiying Chen, Junlong Shen, Zhanyuan Guo, Xiaoou Zhou

arXiv 2607.02802首次发表:更新:

发表机构

University of Alberta; Qilu University of Technology; N-Dice Association(阿尔伯塔大学; 齐鲁工业大学; N-Dice协会)

机构由 AI 辅助整理,请以论文原文为准。

AI 中文总结

研究在半开放文本游戏环境中,当用户意图与系统规则冲突时大语言模型的规则遵守问题。基于桌面角色扮演游戏机制构建多智能体对抗基准CoC-Seduce,用前沿模型生成样本,对20个目标裁决器进行基准测试。

AI 中文摘要

随着大语言模型越来越多地在半开放文本游戏环境中作为自主裁决器部署,当用户意图与系统规则冲突时,强大的规则遵守变得至关重要。然而,这些模型容易受到一类我们称为“修辞注入”的攻击。我们提出CoC-Seduce,一个基于桌面角色扮演游戏机制的多智能体对抗基准,并对20个目标裁决器进行基准测试。

英文摘要

As LLMs are increasingly deployed as autonomous adjudicators in games such as Call of Cthulhu (CoC), robust rule adherence becomes critical when user intent conflicts with system rules. However, as these models are trained to be helpful and compliant, they may be vulnerable to a class of manipulations we term Rhetorical Injection, where adversarial users exploit narrative framing techniques such as pseudo-logical reasoning and authoritative coercion to bypass adjudication logic. We present CoC-Seduce, a multi-agent adversarial benchmark built on CoC, a Tabletop Role-Playing Game (TRPG) in which rules are explicit about which risky actions require adjudication, yet interaction remains entirely in natural language. Three LLMs, i.e., GPT-5.4, Claude Sonnet 4.6, Gemini 3.5 Flash, serve as adversarial generators producing 5,376 samples across 4 world settings and 16 skill categories. We then benchmark 22 target adjudicators against this corpus. Evaluation across 22 models reveals that neither newer releases nor explicit reasoning reliably confer adjudication robustness, that Pseudo-Logic framing is the most effective rhetorical style, and that the world setting, including culturally distant ones, has only a modest effect. Project page: https://github.com/answerrtx/CoC-Seduce.

Commentscorrected errors, added evaluations of new models, and revised the scope of the paper

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文

↑