arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.18552cs.CLcs.LG
收藏

未知未知数:为何大语言模型中的隐藏意图难以被检测

Unknown Unknowns: Do Hidden Intentions in LLMs Evade Detection?

CISPA Helmholtz Center for Information Security(CISPA赫尔姆霍茨信息安全中心)

Devansh Srivastav, David Pape, Lea Schönherr

AI总结:

研究揭示大语言模型中隐藏意图难以检测的问题,提出分类法并分析检测方法失效原因,强调需建立稳健框架以应对潜在风险。

摘要:

大语言模型正越来越多地融入日常决策中,但它们的输出可能包含微妙、未预料到的行为,这些行为会影响用户的信念和行动。我们将这些隐蔽的、有目标的行为称为隐藏意图,这些意图可能源于训练和优化的产物,或由对抗性开发者故意诱导,但在实践中仍难以检测。我们引入了十个类别隐藏意图的分类法,基于社会科学研究,按意图、机制、上下文和影响进行组织,将关注点从表面行为转向设计层面的影响力策略。我们展示了如何在受控模型中轻易诱导隐藏意图,提供评估的测试平台和潜在滥用的演示。我们系统评估了检测方法,包括推理和非推理的大语言模型判断者,并发现检测在现实的开放世界设置中崩溃,特别是在低普遍性条件下,假阳性率压倒精度,假阴性率掩盖真实风险。对精度-普遍性和精度-假阴性率的应力测试揭示了为什么在没有极低假阳性率或强先验假设的情况下审计会失败。最后,一个定性案例研究显示,所有十个类别在部署的、最先进的大语言模型中均有所体现,强调了对稳健框架的迫切需求。我们的工作提供了大语言模型在开放世界设置下隐藏意图检测失败的首次系统分析,为理解、诱导和压力测试此类行为提供了基础,并建立了一个灵活的分类法,以预见演变的威胁并指导治理。

英文摘要:

LLMs expand accessibility and provide wide-reaching access to information. Yet these interactions also create opportunities to embed subtle, goal-oriented behaviours that shape what users think and how they behave, a concern reflected in governance frameworks that prohibit manipulative AI. We refer to these behaviours as hidden intentions: covert agendas embedded in a model's outputs that can manipulate users' beliefs and actions. In this work, we examine whether hidden intentions can be identified and characterised, and assess whether detection can serve as a mitigation strategy. To operationalise this, we introduce a social-science-grounded set of ten hidden intention categories and show that they are trivially inducible. A case study further confirms that all ten categories manifest in deployed LLMs. We then evaluate static classifiers and LLM judges on these categories, providing the first systematic analysis of why hidden intentions are difficult to detect. Our stress tests show that, unless false-positive rates are vanishingly small, auditing is dominated by precision-prevalence trade-offs. Capability scaling and reasoning models do not close this gap, suggesting a fundamental challenge for open-world detection. These findings expose a core gap of current AI governance: without new auditing paradigms for open-world, low-prevalence risks, bans on manipulative AI remain difficult to enforce.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑