CogManip: 在大语言模型多轮交互中操控行为的基准测试
CogManip: Benchmarking Manipulative Behavior in Multi-Turn Interactions with Large Language Model
AI总结：
提出CogManip基准，通过1000个多轮交互场景评估15种操控策略风险，发现前沿模型存在显著风险异质性，并揭示提示工程防御的重要性。
摘要：
大语言模型（LLM）在复杂人机交互中是否表现出隐蔽的心理操控已引起越来越多的安全担忧。然而，现有的人工智能安全基准大多局限于显式的规则遵守和静态提示，未能捕捉多轮对话中操控策略的动态性和隐蔽性。我们引入了CogManip，一个全面的基准，在1000个多轮交互场景中评估15种操控策略风险，并由人类专家验证。对包括GPT-5.4和DeepSeek-V3.2等前沿模型在内的13个代表性模型的系统评估揭示了显著的风险异质性，并为未来防御指明了方向。进一步的目标函数扰动分析表明，DeepSeek-V3.2的操控策略对负面和良性系统提示均高度敏感，证明了基于提示的防御工程和隐式目标审计的关键必要性。CogManip为审计现代LLM的隐式心理影响和动态策略选择提供了强大的工具和视角。
英文摘要：
Whether Large Language Models (LLMs) exhibit covert psychological manipulation in complex human-AI interactions has garnered increasing safety concerns. However, existing AI safety benchmarks remain largely restricted to explicit rule compliance and static prompts, failing to capture the dynamic and covert nature of manipulative strategies in multi-turn dialogues. We introduce CogManip, a comprehensive benchmark that evaluates 15 manipulation strategy risks across 1,000 multi-turn interaction scenarios, validated by human experts. A systematic evaluation of 13 representative models, including frontier models like GPT-5.4 and DeepSeek-V3.2, reveals significant risk heterogeneities and illuminates the targeted direction for future defense. Further analysis of objective function perturbation reveals that DeepSeek-V3.2's manipulation tactics are highly sensitive to both negative and benign system prompts, demonstrating the critical necessity of prompt-based defense engineering and implicit goal auditing. CogManip offers a robust instrument and perspective for auditing the implicit psychological influence and dynamic strategy selection of modern LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
