arXiv 2504.141172025-04-19cs.CLcs.CV

PEFT A2Z：大型语言与视觉模型的参数高效微调综述

PEFT A2Z: Parameter-Efficient Fine-Tuning Survey for Large Language and Vision Models

University of Central Florida（佛罗里达中央大学）；Khulna University of Engineering & Technology（库尔纳工程与技术大学）；Delineate Inc.（Delineate公司）；Universiti Tenaga Nasional（国家能源大学）；University of South Florida（佛罗里达州立大学）；Daffodil International University（达夫迪尔国际大学）；Anymate Me（Anymate Me公司）

Nusrat Jahan Prottasha, Upama Roy Chowdhury, Shetu Mohanto, Tasfia Nuzhat, Abdullah As Sami, Md Shamol Ali, Md Shohanur Islam Sobuj, Hafijur Raman, Md Kowsher, Ozlem Ozmen Garibay

备注：: PEFT Survey paper

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文综述了参数高效微调（PEFT）技术，提出分类法比较各类方法，并探讨其在语言、视觉等领域的应用及未来挑战，旨在实现大型模型的高效可持续使用。

摘要：

大型语言模型（LLMs）和视觉语言模型（VLMs）等大型模型已彻底改变了人工智能，推动了自然语言处理、计算机视觉和多模态学习中的应用。然而，完全微调这些模型仍然代价高昂，需要大量的计算资源、内存和特定任务的数据。参数高效微调（PEFT）已成为一种有前景的解决方案，允许通过仅更新一小部分参数来使大型模型适应下游任务。本综述全面概述了PEFT技术，重点关注其动机、设计原则和有效性。我们首先分析了传统微调带来的资源和可访问性挑战，并强调了过拟合、灾难性遗忘和参数低效等关键问题。然后，我们引入了PEFT方法的结构化分类法——分为加法式、选择式、重参数化式、混合式和统一式框架——并系统地比较了它们的机制和权衡。在分类法之外，我们探讨了PEFT在语言、视觉和生成建模等不同领域的影响，展示了这些技术如何以较低的资源成本提供强大的性能。我们还讨论了可扩展性、可解释性和鲁棒性方面的重要开放挑战，并提出了未来方向，如联邦学习、领域适应和理论基础。我们的目标是提供对PEFT及其在实现大型模型实用、高效和可持续使用中日益增长作用的统一理解。

英文摘要：

Large models such as Large Language Models (LLMs) and Vision Language Models (VLMs) have transformed artificial intelligence, powering applications in natural language processing, computer vision, and multimodal learning. However, fully fine-tuning these models remains expensive, requiring extensive computational resources, memory, and task-specific data. Parameter-Efficient Fine-Tuning (PEFT) has emerged as a promising solution that allows adapting large models to downstream tasks by updating only a small portion of parameters. This survey presents a comprehensive overview of PEFT techniques, focusing on their motivations, design principles, and effectiveness. We begin by analyzing the resource and accessibility challenges posed by traditional fine-tuning and highlight key issues, such as overfitting, catastrophic forgetting, and parameter inefficiency. We then introduce a structured taxonomy of PEFT methods -- grouped into additive, selective, reparameterized, hybrid, and unified frameworks -- and systematically compare their mechanisms and trade-offs. Beyond taxonomy, we explore the impact of PEFT across diverse domains, including language, vision, and generative modeling, showing how these techniques offer strong performance with lower resource costs. We also discuss important open challenges in scalability, interpretability, and robustness, and suggest future directions such as federated learning, domain adaptation, and theoretical grounding. Our goal is to provide a unified understanding of PEFT and its growing role in enabling practical, efficient, and sustainable use of large models.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。