ADVERSA: Measuring Multi-Turn Guardrail Degradation and Judge Reliability in Large Language Models
Independent Researcher(独立研究者)
Harry Owiredu-Ashley
备注:
12 pages, 12 figures. Independent research. Code and artifacts: https://github.com/Harry-Ashley/adversa-guardrail-degradation. Published in the 2026 IEEE/ACIS 24th International Conference on Software Engineering Research, Management and Applications (SERA), pp. 414-417. Recipient of the Best Paper Award at IEEE/ACIS SERA 2026
发表信息:
2026 IEEE/ACIS 24th International Conference on Software Engineering Research, Management and Applications (SERA), Towson, MD, USA, 2026, pp. 414-417
大多数针对大型语言模型(LLM)安全性的对抗性评估仅评估单个提示并报告二元通过/失败结果,这无法捕捉到在持续对抗交互下安全属性如何演变。我们提出了ADVERSA,一个自动红队框架,该框架通过连续的每轮合规轨迹来测量护栏退化动态,而不是离散的 jailbreak 事件。ADVERSA 使用了一个微调过的70B攻击者模型(ADVERSA-Red,Llama-3.1-70B-Instruct with QLoRA),该模型消除了攻击方侧的安全拒绝,这些拒绝使得现成的模型在作为攻击者时不可靠。该框架对受害者响应进行评分,使用一个结构化的5点评分表,将部分合规视为一个可测量的状态。我们报告了在三个前沿受害者模型(Claude Opus 4.6、Gemini 3.1 Pro、GPT-5.2)上进行的受控实验,使用三裁判共识架构,其中裁判可靠性被作为第一类研究结果来测量,而不是假设。在15次最多10轮对抗交互的对话中,我们观察到26.7%的jailbreak率,平均jailbreak轮次为1.25,表明在该评估设置中,成功的jailbreak集中在早期轮次,而不是通过持续压力积累。我们记录了裁判间的一致性比率、自我评分倾向、攻击者漂移作为微调攻击者部署出其训练分布时的故障模式,以及攻击者拒绝作为之前未报告的混淆因素在受害者抵抗测量中的影响。所有限制都明确陈述。攻击提示根据负责任的披露政策保留;所有其他实验 artifacts 都已发布。
英文摘要:
Most adversarial evaluations of large language model (LLM) safety assess single prompts and report binary pass/fail outcomes, which fails to capture how safety properties evolve under sustained adversarial interaction. We present ADVERSA, an automated red-teaming framework that measures guardrail degradation dynamics as continuous per-round compliance trajectories rather than discrete jailbreak events. ADVERSA uses a fine-tuned 70B attacker model (ADVERSA-Red, Llama-3.1-70B-Instruct with QLoRA) that eliminates the attacker-side safety refusals that render off-the-shelf models unreliable as attackers, scoring victim responses on a structured 5-point rubric that treats partial compliance as a distinct measurable state.
We report a controlled experiment across three frontier victim models (Claude Opus 4.6, Gemini 3.1 Pro, GPT-5.2) using a triple-judge consensus architecture in which judge reliability is measured as a first-class research outcome rather than assumed. Across 15 conversations of up to 10 adversarial rounds, we observe a 26.7% jailbreak rate with an average jailbreak round of 1.25, suggesting that in this evaluation setting, successful jailbreaks were concentrated in early rounds rather than accumulating through sustained pressure. We document inter-judge agreement rates, self-judge scoring tendencies, attacker drift as a failure mode in fine-tuned attackers deployed out of their training distribution, and attacker refusals as a previously-underreported confound in victim resistance measurement. All limitations are stated explicitly. Attack prompts are withheld per responsible disclosure policy; all other experimental artifacts are released.
HE-Guardrail: A Homomorphic Guardrail Against Jailbreak Attacks for Encrypted Large Language Model Inference
Pohang University of Science and Technology (POSTECH)(浦项科技大学);Inha University(仁荷大学);Daegu Gyeongbuk Institute of Science and Technology (DGIST)(大邱庆北科学技术院)
HoloAegis: Frozen Representation, Topological Inference --- Minimally Parametric Safety Manifolds and Their Capability Boundaries for LLM Guardrails
Hong Kong Baptist University(香港浸会大学);Guangdong Polytechnic Normal University(广东技术师范大学);Guangdong Institute of Digital Industry(广东数字产业研究院);The Hong Kong Polytechnic University(香港理工大学);The Education University of Hong Kong(香港教育大学);Southern University of Science and Technology(南方科技大学)
RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content
University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Virginia Tech(弗吉尼亚理工大学);Salesforce Research(Salesforce研究院);University of California Berkeley(加州大学伯克利分校);University of Chicago(芝加哥大学)
Privacy Policy Enforcement Guardrails for Data-Sensitive Retrieval-Augmented Generation
University of California, Berkeley(加州大学伯克利分校);University of Washington(华盛顿大学);University of Toronto(多伦多大学);University of Texas at Austin(德克萨斯大学奥斯汀分校);University of California, Los Angeles(加州大学洛杉矶分校)
SafeHarbor: Defining Precise Decision Boundaries via Hierarchical Memory-Augmented Guardrail for LLM Agent Safety
School of Cyber Science and Technology, Beihang University, Beijing, China(北京航空航天大学网络安全学院);Institute of Artificial Intelligence, Beihang University, Beijing, China(北京航空航天大学人工智能研究院);University of Chinese Academy of Sciences, Beijing, China(中国科学院大学);AI Security Lab, Beijing, China(360人工智能安全实验室)
Risk-Aware LLM Agents for Geospatial Data Retrieval: Design and Preliminary Adversarial Evaluation
Dept. of Systems Design Engineering, University of Waterloo(滑铁卢大学系统设计工程系);SkyWatch;Dept. of Geography and Environmental Management, University of Waterloo(滑铁卢大学地理与环境管理系);Dept. of Geomatics Engineering, University of Calgary(卡尔加里大学测绘工程系)
Reflex-Guard: A Low-Latency Guardrail for LLM Prompt Safety Using Dense Semantic Embeddings
Graduate School of Informatics, Osaka Metropolitan University(大阪公立大学情报学研究科);BRAC University(BRAC大学);Bangladesh University of Engineering and Technology (BUET)(孟加拉工程技术大学);New Mexico State University(新墨西哥州立大学)