发表机构
Richter Optimization GmbH; Sony AI（里希特优化有限公司; 索尼人工智能）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对操作受约束动态系统，提出FAOC框架，整合RL与OC。通过计算高效映射算法，将RL动作转换为OCP可行参数集，兼顾安全性与灵活性。应用于机器人乒乓球运动规划，模拟实验显示其在样本效率和闭环性能上优于现有基准。
AI 中文摘要
操作受约束的动态系统要求控制器在执行递归可行性和安全约束时有效解决复杂任务。为满足这些相互竞争的要求，我们提出了最优控制的可行动作（FAOC），这是一个整合强化学习（RL）和最优控制（OC）的新型控制框架。关键贡献是一种基于优化的计算高效映射算法，它将RL智能体的动作从静态抽象集转换为最优控制问题（OCP）的状态依赖可行参数集，确保严格满足动态系统约束。因此，FAOC有效地将OC的可预测安全性与RL的灵活性结合起来。与先前工作不同，RL智能体的抽象动作空间无需专家或启发式设计，且OCP公式不会因RL无法保证可行性而受损。我们将该方法应用于机器人乒乓球实时运动规划，通过模拟实验表明，FAOC在样本效率和闭环性能方面均优于现有基准。
英文摘要
Operating constrained dynamical systems requires controllers to efficiently solve complex tasks while enforcing recursive feasibility and physical constraints. To address these competing requirements, we present Feasible Action for Optimal Control (FAOC), a novel control framework integrating Reinforcement Learning (RL) and Optimal Control (OC). The core contribution is a computationally efficient, optimization-based mapping algorithm that transforms the RL agent's action from a static abstract set into a state-dependent feasible parameter set of the Optimal Control Problem (OCP), guaranteeing instantaneous parameter feasibility. When paired with invariant terminal sets, FAOC guarantees strict recursive feasibility and safe operation, effectively combining the predictable safety of OC with the behavioral flexibility of RL. Unlike prior work, the abstract action space does not require expert tuning, nor is the OCP formulation compromised by the inability of RL to guarantee feasibility. We evaluate FAOC on real-time motion planning for robot table tennis, where simulated experiments demonstrate superior sample efficiency and closed-loop performance compared to state-of-the-art baselines. We open-source the used implementation of the mapping algorithm and OCP for motion planning https://github.com/SonyResearch/feasible_action_for_optimal_control.
Comments26 pages, 6 figures
