Parameter-Efficient Fine-Tuning (PEFT) has emerged as a practical paradigm for adapting large language models (LLMs) without updating all parameters. Most existing approaches, such as LoRA and PiSSA, rely on low-rank decompositions of weight updates. However, the low-rank assumption may restrict expressivity, particularly in task-specific adaptation scenarios where singular values are distributed relatively uniformly. To address this limitation, we propose CoSA (Compressed Sensing-Based Adaptation), a new PEFT method extended from compressed sensing theory. Instead of constraining weight updates to a low-rank subspace, CoSA expresses them through fixed random projection matrices and a compact learnable core. We provide a formal theoretical analysis of CoSA as a synthesis process, proving that weight updates can be compactly encoded into a low-dimensional space and mapped back through random projections. Extensive experimental results show that CoSA provides a principled perspective for efficient and expressive multi-scale model adaptation. Specifically, we evaluate CoSA on 10 diverse tasks, including natural language understanding and generation, employing 5 models of different scales from RoBERTa, Llama, and Qwen families. Across these settings, CoSA consistently matches or outperforms state-of-the-art PEFT methods.
DIB-OD: Preserving the Invariant Core for Robust Heterogeneous Graph Adaptation via Decoupled Information Bottleneck and Online Distillation
School of Information Technology and Engineering, Tianjin University of Technology and Education（天津职业技术师范大学信息技术工程学院）；School of Computer Science and Technology, Tiangong University（天工大学计算机科学与技术学院）；Department of Computer Science at University of Exeter（埃克塞特大学计算机科学系）
TSFLora: Token-Compressed Split Fine-Tuning for Wireless Edge Networks
School of Computer Science and Engineering（计算机科学与工程学院）；Center for Intelligent Networking and Communications（智能网络与通信中心）；School of Computer Science（计算机科学学院）；Beijing University of Posts and Telecommunications（北京邮电大学）
