arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.24445cs.LGcs.SYeess.SY
收藏

由偏倚-噪声对齐诊断引导的自适应学习

Adaptive Learning Guided by Bias-Noise-Alignment Diagnostics

Department of Electrical, Computer and Software Engineering, Ontario Tech University(电气、计算机与软件工程系, Ontario Tech 大学)

Akash Samanta, Sheldon Williamson

AI总结:

本文提出了一种由偏倚-噪声对齐诊断引导的自适应学习框架,通过分解误差演变的三个组成部分,为监督优化、强化学习和学习优化器提供统一控制核心,并展示了其在动态环境中的稳定性和有效性。

摘要:

在非平稳和安全关键环境中部署的学习系统常常在动态演变时出现不稳定、收敛缓慢或脆弱适应的问题。尽管现代优化、强化学习和元学习方法能够适应梯度统计信息,但它们大多忽略了误差信号本身的时间结构。本文提出了一种以诊断为导向的自适应学习框架,通过系统性地将误差演变分解为偏倚、噪声和对齐三个部分来显式建模误差演变。偏倚捕捉持续漂移,噪声捕捉随机变异性,对齐捕捉重复方向激励导致的超调。这些诊断是通过轻量级的损失或时序差分(TD)误差轨迹统计信息在线计算的,并且与模型架构或任务领域无关。我们展示了所提出的偏倚-噪声-对齐分解为监督优化、actor-critic强化学习和学习优化器提供了统一的控制核心。在此框架内,我们引入了三种诊断驱动的实例:受人类启发的监督自适应优化器(HSAO)、用于actor-critic方法的混合误差-诊断强化学习(HED-RL)以及元学习学习策略(MLLP)。在标准光滑性假设下,我们为所有情况建立了有界有效更新和稳定性属性。在actor-critic学习中的代表性诊断示例突显了所提出信号如何调节适应以响应TD误差结构。总体而言,这项工作将误差演变提升为自适应学习中的首要对象,并提供了一个可解释、轻量级的基础,以在动态环境中实现可靠的学习。

英文摘要:

Learning systems deployed in nonstationary and safety-critical environments often suffer from instability, slow convergence, or brittle adaptation when learning dynamics evolve over time. While modern optimization, reinforcement learning, and meta-learning methods adapt to gradient statistics, they largely ignore the temporal structure of the error signal itself. This paper proposes a diagnostic-driven adaptive learning framework that explicitly models error evolution through a principled decomposition into bias, capturing persistent drift; noise, capturing stochastic variability; and alignment, capturing repeated directional excitation leading to overshoot. These diagnostics are computed online from lightweight statistics of loss or temporal-difference (TD) error trajectories and are independent of model architecture or task domain. We show that the proposed bias-noise-alignment decomposition provides a unifying control backbone for supervised optimization, actor-critic reinforcement learning, and learned optimizers. Within this framework, we introduce three diagnostic-driven instantiations: the Human-inspired Supervised Adaptive Optimizer (HSAO), Hybrid Error-Diagnostic Reinforcement Learning (HED-RL) for actor-critic methods, and the Meta-Learned Learning Policy (MLLP). Under standard smoothness assumptions, we establish bounded effective updates and stability properties for all cases. Representative diagnostic illustrations in actor-critic learning highlight how the proposed signals modulate adaptation in response to TD error structure. Overall, this work elevates error evolution to a first-class object in adaptive learning and provides an interpretable, lightweight foundation for reliable learning in dynamic environments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑