迈向可信星载AI：利用强化学习推进小卫星运行
Toward Trusted Onboard AI: Advancing Small Satellite Operations using Reinforcement Learning
AI总结：
本研究针对3U立方星星载指令自动化需求，开发宏控制动作强化学习算法，通过数字孪生体训练智能体，在隔离环境中无指令权限验证轨道有效性，为可信AI集成到小卫星星载系统提供了可行路径。
摘要：
本研究开发了一种用于3U CubeSat（立方星）星上指令自动化的RL（Reinforcement Learning，强化学习）算法。研究聚焦于宏控制动作强化学习的实现，该技术为星上智能体提供基于实时遥测数据的编译信息作为观测输入。智能体利用这些信息生成高层动作，例如将姿态调整为太阳指向，这些动作随后会被转换为控制算法，并通过低层指令执行。一旦建立起对星上智能体的信任，便可利用实时环境信息实现更快的响应速度，并降低对地面控制的依赖。该方法不仅专注于为特定卫星开发强化学习算法，还为将可信AI集成到星载系统树立了先例。\n本研究基于三个领域的已有工作展开：（1）用于生成可转换为低层可执行指令的高层命令的强化学习算法；（2）与实时运行系统（尤其是航天器星载系统）对接的AI推理模型部署；（3）AI系统的信任构建策略，特别是针对远程和自主应用的策略。现有的卫星控制强化学习研究大多局限于基于仿真的实验；在本工作中，研究人员通过构建特定航天器的数字孪生体，并在该仿真环境中训练强化学习智能体生成宏动作，对这些技术进行了定制化改进。训练完成的智能体策略被复制到一个隔离环境中，向其输入卫星的编译信息以进行推理预测，从而在不授予其指令权限的情况下验证该强化学习算法在轨道上的有效性。这一流程能够将算法的预测结果与卫星实际行为进行安全对比，并确保运行处于预期参数范围内。
英文摘要：
A RL (Reinforcement Learning) algorithm was developed for command automation onboard a 3U CubeSat. This effort focused on the implementation of macro control action RL, a technique in which an onboard agent is provided with compiled information based on live telemetry as its observation. The agent uses this information to produce high-level actions, such as adjusting attitude to solar pointing, which are then translated into control algorithms and executed through lower-level instructions. Once trust in the onboard agent is established, real-time environmental information can be leveraged for faster response times and reduced reliance on ground control. The approach not only focuses on developing an RL algorithm for a specific satellite but also sets a precedent for integrating trusted AI into onboard systems. This research builds on previous work in three areas: (1) RL algorithms for issuing high-level commands that are translated into low-level executable instructions; (2) the deployment of AI inference models interfaced with live operational systems, particularly onboard spacecraft; and (3) strategies for building trust in AI systems, especially for remote and autonomous applications. Existing RL research for satellite control is largely limited to simulation-based experiments; in this work, these techniques are tailored by constructing a digital twin of a specific spacecraft and training the RL agent to issue macro actions in this simulated environment. The policy of the trained agent is copied to an isolated environment, where it is fed compiled information about the satellite to make inference predictions, thereby demonstrating the RL algorithm's validity on orbit without granting it command authority. This process enables safe comparison of the algorithm's predictions against actual satellite behavior and ensures operation within expected parameters.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
