对抗性道德压力测试：大语言模型的伦理鲁棒性评估
Adversarial Moral Stress Testing of Large Language Models
AI总结：
本文提出AMST框架，通过对抗性多轮交互评估大语言模型的伦理鲁棒性，揭示传统单轮评估无法发现的降级模式，强调分布稳定性与尾部行为的重要性。
摘要：
评估部署在软件系统中的大语言模型（LLM）的伦理鲁棒性仍然具有挑战性，特别是在持续的对抗性用户交互下。现有安全基准通常依赖单轮评估和聚合指标，如毒性评分和拒绝率，这些指标对现实多轮交互中可能出现的行为不稳定性提供有限的可见性。因此，在部署前，罕见但高影响的伦理失败和渐进性降级效应可能未被检测到。本文介绍了对抗性道德压力测试（AMST），一种基于压力的评估框架，用于在对抗性多轮交互下评估伦理鲁棒性。AMST对提示应用结构化压力转换，并通过分布感知的鲁棒性指标评估模型行为，这些指标捕捉了交互轮次中的方差、尾部风险和时间行为漂移。我们评估了AMST在多个最先进的LLM上，包括LLaMA-3-8B、GPT-4o和DeepSeek-v3，使用大量在受控压力条件下生成的对抗性场景。结果表明，不同模型在鲁棒性特征上存在显著差异，并揭示了在传统单轮评估协议下不可观测的降级模式。特别是，鲁棒性已被证明取决于分布稳定性与尾部行为，而非仅平均性能。此外，AMST提供了一种可扩展且模型无关的压力测试方法，使LLM启用的软件系统在对抗性环境中能够进行鲁棒性感知的评估和监控。
英文摘要：
Evaluating the ethical robustness of large language models (LLMs) deployed in software systems remains challenging, particularly under sustained adversarial user interaction. Existing safety benchmarks typically rely on single-round evaluations and aggregate metrics, such as toxicity scores and refusal rates, which offer limited visibility into behavioral instability that may arise during realistic multi-turn interactions. As a result, rare but high-impact ethical failures and progressive degradation effects may remain undetected prior to deployment. This paper introduces Adversarial Moral Stress Testing (AMST), a stress-based evaluation framework for assessing ethical robustness under adversarial multi-round interactions. AMST applies structured stress transformations to prompts and evaluates model behavior through distribution-aware robustness metrics that capture variance, tail risk, and temporal behavioral drift across interaction rounds. We evaluate AMST on several state-of-the-art LLMs, including LLaMA-3-8B, GPT-4o, and DeepSeek-v3, using a large set of adversarial scenarios generated under controlled stress conditions. The results demonstrate substantial differences in robustness profiles across models and expose degradation patterns that are not observable under conventional single-round evaluation protocols. In particular, robustness has been shown to depend on distributional stability and tail behavior rather than on average performance alone. Additionally, AMST provides a scalable and model-agnostic stress-testing methodology that enables robustness-aware evaluation and monitoring of LLM-enabled software systems operating in adversarial environments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
