arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.24152cs.AI
收藏

神经启发式逆向学习用于规划与控制

Neuro-Inspired Inverse Learning for Planning and Control

NeuroMentum AI IMBIT, University of Freiburg, Germany(NeuroMentum AI IMBIT,弗赖堡大学,德国)

Maryna Kapitonova, Tonio Ball

AI总结:

提出一种神经启发式框架Inverter,通过逆向学习(IL)结合前向/逆向内部模型、开环多步运动指令和层次化动作组织,在规划与控制任务中实现高效推理,平均性能提升24.2%且计算时间降低一到两个数量级。

摘要:

我们提出了一种用于具身规划与控制的神经启发式框架。基于哺乳动物大脑中实现快速高效目标导向行为的三个原则——配对的前向/逆向内部模型、开环多步运动指令以及顺序层次化的动作组织——我们的Inverter框架使用学习组件,通过逆向学习(IL)进行端到端训练,并在自然情况下辅以解析或算法模块;我们形式化了IL,并将其与监督学习、强化学习和模仿学习区分开来。IL桥接了强化学习(RL)式的摊销(单次前向传播但每次只输出一个动作)和最优控制(OC)式的序列规划(整个轨迹但需要迭代测试时计算)。单个Inverter或层次化n=2的Inverter堆栈在所有3个maze2d和6个antmaze D4RL变体上,平均比离线RL和扩散规划基线提升24.2%(范围-1.9%至+78.2%),同时推理计算时间减少一到两个数量级。显著的是,通过前向模型(FoM)对整个T步动作序列进行优化(而非逐步骤优化),使得Inverter能够生成平滑、目标一致、轨迹级的结构,并达到比训练数据本身所蕴含的策略更接近解析最优的控制策略。我们还发现了IL的一种失败模式:在训练数据覆盖范围狭窄时出现FoM攻击,我们通过使用覆盖范围更广的随机训练数据来缓解。作为一个应用实例,脉冲Inverter合成任意单量子比特量子门,其保真度与标准迭代数值基线(GRAPE)相当,而每个门的计算时间降低超过1000倍。总之,我们得出结论:IL实现了一类通用的世界接口,特别适用于对延迟和资源敏感的具身AI。

英文摘要:

We present a neuro-inspired framework for embodied planning and control. Building on three principles that enable fast and highly effective goal-directed behavior in the mammalian brain - paired forward/inverse internal models, open-loop multi-step motor commands, and sequential, hierarchical organization of action - our Inverter framework uses learned components, trained end-to-end through Inverse Learning (IL) and supplemented where natural by analytic or algorithmic modules; we formalize IL and delineate it from supervised, reinforcement, and imitation learning. IL bridges Reinforcement Learning (RL)-style amortization, which runs in a single forward pass but emits only one action at a time, and Optimal Control (OC)-style sequence planning over whole trajectories, but with iterative test-time computation. Single Inverters or hierarchical n=2 Inverter stacks match or improve on offline-RL and diffusion-planner baselines on all 3 maze2d and 6 antmaze D4RL variants by an average of +24.2% (range -1.9% to +78.2%), at one-to-two orders of magnitude less inference compute time. Distinctively, optimizing through the Forward Model (FoM) over the entire T-step action sequence - rather than per step - lets Inverters produce smooth, goal-coherent, trajectory-wide structure and reach control policies closer to the analytic optimum than the policy underlying the training data itself. We also identify a failure mode of IL: FoM hacking under narrow training-data coverage, which we mitigate by using random training data with broader coverage. As an application example, a Pulse Inverter synthesizes arbitrary single-qubit quantum gates with fidelity matching the standard iterative numerical baseline (GRAPE), at more than 1000x lower per-gate compute time. In summary, we conclude that IL enables a versatile class of world-interfaces, especially for latency- and resource-critical embodied AI.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑