预期危害：重新思考对（误）对齐大语言模型的安全性评估
Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs
AI总结：
本文提出预期危害指标，通过分析发现模型对高成本低可能性威胁的拒绝行为与高可能性低成本威胁的易受攻击现象，揭示模型对执行成本的
摘要：
当前对大语言模型（LLM）安全性的评估主要依赖于基于严重性的分类体系来评估恶意查询的危害性。我们认为这种框架需要重新审视，因为它假设所有恶意查询具有相同的风险，忽视了执行可能性——即在模型响应条件下威胁得以实现的条件概率。在本工作中，我们引入了预期危害这一指标，该指标将威胁的严重性与其执行可能性相结合，模型为执行成本的函数。通过实证分析最先进模型，我们揭示了一个系统性的逆风险校准：模型对低可能性（高成本）威胁表现出更强的拒绝行为，而对高可能性（低成本）查询却仍易受攻击。我们证明这种校准错误会创建结构性漏洞：通过利用这一属性，我们能够将现有劫持攻击的成功率提高多达2倍。最后，我们通过线性探测追溯这一失败的根本原因，发现虽然模型在潜在空间中编码了严重性以驱动拒绝决策，但它们在执行成本方面没有可区分的内部表示，使它们
英文摘要：
Current evaluations of LLM safety predominantly rely on severity-based taxonomies to assess the harmfulness of malicious queries. We argue that this formulation requires re-examination as it assumes uniform risk across all malicious queries, neglecting Execution Likelihood--the conditional probability of a threat being realized given the model's response. In this work, we introduce Expected Harm, a metric that weights the severity of a jailbreak by its execution likelihood, modeled as a function of execution cost. Through empirical analysis of state-of-the-art models, we reveal a systematic Inverse Risk Calibration: models disproportionately exhibit stronger refusal behaviors for low-likelihood (high-cost) threats while remaining vulnerable to high-likelihood (low-cost) queries. We demonstrate that this miscalibration creates a structural vulnerability: by exploiting this property, we increase the attack success rate of existing jailbreaks by up to $2\times$. Finally, we trace the root cause of this failure using linear probing, which reveals that while models encode severity in their latent space to drive refusal decisions, they possess no distinguishable internal representation of execution cost, making them "blind" to this critical dimension of risk.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
