As frontier AIs become more powerful and costly to develop, adversaries have increasing incentives to steal model weights by mounting exfiltration attacks. In this work, we consider exfiltration attacks where an adversary attempts to sneak model weights out of a datacenter over a network. While exfiltration attacks are multi-step cyber attacks, we demonstrate that a single factor, the compressibility of model weights, significantly heightens exfiltration risk for large language models (LLMs). We tailor compression specifically for exfiltration by relaxing decompression constraints and demonstrate that attackers could achieve 16x to 100x compression with minimal trade-offs, reducing the time it would take for an attacker to illicitly transmit model weights from the defender's server from months to days. Finally, we study defenses designed to reduce exfiltration risk in three distinct ways: making models harder to compress, making them harder to 'find,' and tracking provenance for post-attack analysis using forensic watermarks. While all defenses are promising, the forensic watermark defense is both effective and cheap, and therefore is a particularly attractive lever for mitigating weight-exfiltration risk.
Recover-LoRA for Aggressive Quantization: Reclaiming Accuracy in 2-Bit Language Models via Low-Rank Adaptation with Knowledge Distillation on Synthetic Data
SWAN: Sparse Winnowed Attention for Reduced Inference Memory via Decompression-Free KV-Cache Compression
Centre for Responsible AI(负责任人工智能中心);Indian Institute of Technology Madras(印度理工学院马德拉斯分校);Department of Electrical Engineering(电气工程系);Wadhwani School of Data Science and Artificial Intelligence(Wadhwani数据科学与人工智能学院)
College of Computer Science and Technology, National University of Defense Technology(国防科技大学计算机科学与技术学院);School of Computer Science, Beijing University of Posts and Telecommunications(北京邮电大学计算机学院)
SpikeMLLM: Spike-based Multimodal Large Language Models via Modality-Specific Temporal Scales and Temporal Compression
1 Institute of Automation, Chinese Academy of Sciences 2 University of Chinese Academy of Sciences 3 Beijing Academy of Artificial Intelligence 4 Zhongguancun Academy 5 Peking University 6 Key Laboratory of Brain Cognition;Brain-inspired Intelligence Technology 7 Spiking Intelligence Lab, Tianqiao \& Chrissy Chen Institute [0.5em] Equal contribution Corresponding authors