发表机构
MMLA-org; Communication University of China (CUC)（MMLA-org; 中国传媒大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出低秩自适应残差连接（LARC），为冻结模型添加紧凑可学习状态，通过反馈梯度步骤显著降低程序选择错误，并区分残差容量与适应效果。
AI 中文摘要
低秩自适应残差连接（LARC）为冻结模型提供了一个紧凑的数值状态，使其能够从反馈中学习。映射 $h+BAh$ 向隐藏表示添加一个低秩校正。一个慢状态 $\rho$ 跨任务学习起始因子；一个私有快状态 $\Phi$ 复制这些因子，随反馈变化，并重置为训练后的初始化。本报告在记忆介导学习架构中指定了数值策略载体的输入侧实现，并考察其因子空间动力学和学习寿命。我们在冻结的MiniCPM5-1B-SFT基底上研究了一个具有12,288个可训练参数的秩-4输入残差。在一个四候选程序选择任务中，相对于重置为各自训练后的静态初始化和适应后初始化，两个反馈梯度步骤将预期查询执行错误分别降低了24.65和36.65个百分点。这些开发结果覆盖16个参数组和三个配对训练种子。一个直接的支撑损失选择规则更加准确，达到0.78125%的错误率。在仓库平衡的公共持续集成作业时间顺序回放中，保留在线更新使半Brier损失从0.1274上升到0.1808。一个固定的后续干预记录了同批次非下降和不一致的未来收益来自缩减更新。综合来看，代数和测量区分了残差容量、相对于起点的适应以及后续决策中的有用性。
英文摘要
Low-Rank Adaptive Residual Connections (LARC) give a frozen model a compact numerical state that can learn from feedback. The map $h+BAh$ adds a low-rank correction to a hidden representation. A slow state $ρ$ learns starting factors across tasks; a private fast state $Φ$ copies them, changes with feedback, and resets to the trained initialization. This report specifies an input-side realization of the numerical policy carrier in Memory-Mediated Learning Architecture and examines its factor-space dynamics and learning lifetime. We study a rank-4 input residual with 12,288 trainable parameters on a frozen MiniCPM5-1B-SFT substrate. In a four-candidate program-selection task, two feedback-gradient steps reduce expected query execution error by 24.65 and 36.65 percentage points relative to resetting to the respective trained static and post-adaptation initializations. These development results cover 16 parameter groups and three paired training seeds. A direct support-loss selection rule is much more accurate, reaching 0.78125% error. In a repository-balanced chronological replay of public continuous-integration jobs, retaining online updates raises half-Brier loss from 0.1274 to 0.1808. A fixed follow-up intervention records same-batch non-descent and inconsistent future benefit from shrinking updates. Together, the algebra and measurements distinguish residual capacity, adaptation relative to a starting point, and usefulness on later decisions.
Comments19 pages, 6 figures, 15 tables. Technical report of MMLA. The authors contributed equally
