Stakeholders -- from model developers to policymakers -- seek to minimize the dual-use risks of large language models (LLMs). An open challenge to this goal is whether technical safeguards can impede the misuse of LLMs, even when models are customizable via fine-tuning or when model weights are fully open. In response, several recent studies have proposed methods to produce durable LLM safeguards for open-weight LLMs that can withstand adversarial modifications of the model's weights via fine-tuning. This holds the promise of raising adversaries' costs even under strong threat models where adversaries can directly fine-tune model weights. However, in this paper, we urge for more careful characterization of the limits of these approaches. Through several case studies, we demonstrate that even evaluating these defenses is exceedingly difficult and can easily mislead audiences into thinking that safeguards are more durable than they really are. We draw lessons from the evaluation pitfalls that we identify and suggest future research carefully cabin claims to more constrained, well-defined, and rigorously examined threat models, which can provide more useful and candid assessments to stakeholders.
BELLS-O: Evaluating the Operational Trade-offs of LLM Supervision Systems
University of Graz, Graz, Austria(格拉茨大学);Supervised Program for Alignment Research (SPAR)(对齐研究监督计划 (SPAR));Centre pour la Sécurité de l'IA (CeSIA), Paris, France(人工智能安全研究中心 (CeSIA),巴黎,法国)
KC-Bench: A Dynamic Interactive Benchmark for Evaluating Knowledge Conflicts in LLM Agents
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The University of Hong Kong(香港大学);Xiamen University Malaysia(马来西亚厦门大学);Beijing University of Posts and Telecommunications(北京邮电大学)
PricingLogic: Evaluating LLMs Reasoning on Complex Tourism Pricing Tasks
Hunan University(湖南大学);Saarland University, Saarland Informatics Campus(萨尔兰大学,萨尔兰信息技术校区);Ningbo Key Laboratory of Spatial Intelligence and Digital, Derivative Institute of Digital Twin, EIT(宁波空间智能与数字关键实验室,数字孪生衍生研究所,EIT);Department of Computing, The Hong Kong Polytechnic University(计算系,香港理工大学)
Exploring the Privacy Protection Capabilities of Chinese Large Language Models
School of Computer Science and Technology, Beijing Jiaotong University(北京交通大学计算机与信息技术学院);Beijing Key Lab of Traffic Data Analysis and Mining, Beijing Jiaotong University(北京交通大学交通数据分析与挖掘北京市重点实验室);Key Laboratory of Big Data Artificial Intelligence in Transportation(Beijing Jiaotong University), Ministry of Education(交通大数据人工智能教育部重点实验室(北京交通大学))
DrugBench: Evaluating AI Control Protocols for Medication Harm Mitigation
Universidad de Buenos Aires, Facultad de Ciencias Exactas y Naturales, Departamento de Computación(布宜诺斯艾利斯大学,精确与自然科学学院,计算机系);AI Safety Argentina (AISAR)(阿根廷人工智能安全组织 (AISAR));Department of Computer Science, University of Oxford(牛津大学计算机科学系);CONICET-Universidad de Buenos Aires, Instituto de Ciencias de la Computación (ICC)(阿根廷国家科学与技术研究理事会-布宜诺斯艾利斯大学,计算机科学研究所 (ICC))
Towards Reliable Machine Translation: Scaling LLMs for Critical Error Detection and Safety
Rheinische Friedrich-Wilhelms-Universität Bonn, Bonn, Germany(莱茵-威斯巴登大学波恩分校);Fraunhofer IAIS, Sankt Augustin, Germany(弗劳恩霍夫人工智能研究所);Lamarr Institute for Machine Learning(拉马尔人工智能与机器学习研究所)
The Art of (Mis)alignment: How Fine-Tuning Methods Effectively Misalign and Realign LLMs in Post-Training
University of Electronic Science and Technology of China(电子科技大学);Flexera;CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);Nanyang Technological University(南洋理工大学)