站在巨人的肩膀上：通过动态 Logits 融合实现 effortless 的弱到强
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion
AI总结：
针对大模型微调内存开销大的问题，本文提出动态 logit 融合方法，利用多个特定任务小模型在解码时自适应分配权重，将知识直接转移给大模型，在单/多任务中显著缩小性能差距并超越全量微调。
摘要：
针对特定任务应用对大型语言模型进行高效微调势在必行，然而这些模型中海量的参数使得其训练日益充满挑战。尽管已有许多关于有效方法的提出，但在更新过程中，梯度计算仍存在巨大的内存开销。我们能否微调一系列特定任务的小模型，并在无需额外训练的情况下将其知识直接转移给一个大得多的模型？在本文中，我们使用 logit 算术探索弱到强（weak-to-strong）的特化，为直接回答这一问题提供了便利。现有的弱到强方法通常采用静态的知识转移比例，并使用单一的小模型来转移复杂的知识，这导致了次优的性能。为了克服这些限制，我们提出了一种动态 logit 融合方法，该方法与一系列特定任务的小模型协同工作，每个小模型专精于不同的任务。此方法在每个解码步骤中自适应地在这些模型之间分配权重，并通过受 Kullback-Leibler 散度约束的优化问题来学习这些权重。我们在单任务和多任务设置下的各种基准上进行了广泛实验，取得了领先的结果。通过将 7B 模型的专业知识转移到 13B 模型，与 13B 模型的全量微调相比，我们的方法在单任务场景中缩小了 96.4% 的性能差距，在多任务场景中缩小了 86.3% 的性能差距。值得注意的是，我们在未见任务上取得了超越性的表现。此外，我们进一步证明，我们的方法能够毫不费力地将上下文学习（用于单任务）和任务算术（用于多任务场景）整合进来。
英文摘要：
Efficient fine-tuning of large language models for task-specific applications is imperative, yet the vast number of parameters in these models makes their training increasingly challenging. Despite numerous proposals for effective methods, a substantial memory overhead remains for gradient computations during updates. \thm{Can we fine-tune a series of task-specific small models and transfer their knowledge directly to a much larger model without additional training?} In this paper, we explore weak-to-strong specialization using logit arithmetic, facilitating a direct answer to this question. Existing weak-to-strong methods often employ a static knowledge transfer ratio and a single small model for transferring complex knowledge, which leads to suboptimal performance. % To address this, To surmount these limitations, we propose a dynamic logit fusion approach that works with a series of task-specific small models, each specialized in a different task. This method adaptively allocates weights among these models at each decoding step, learning the weights through Kullback-Leibler divergence constrained optimization problems. We conduct extensive experiments across various benchmarks in both single-task and multi-task settings, achieving leading results. By transferring expertise from the 7B model to the 13B model, our method closes the performance gap by 96.4\% in single-task scenarios and by 86.3\% in multi-task scenarios compared to full fine-tuning of the 13B model. Notably, we achieve surpassing performance on unseen tasks. Moreover, we further demonstrate that our method can effortlessly integrate in-context learning for single tasks and task arithmetic for multi-task scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
