ModuLoop: 使用模块化合成器和闭环调试器进行机器人控制的低级代码生成
ModuLoop : Low-Level Code Generation using Modular Synthesizer and Closed-Loop Debugger for Robotic Control
AI总结:
提出闭环模块化代码合成框架,利用预训练大语言模型进行模块化代码规划与生成,并通过迭代执行和调试探针实现系统调试与优化,成功应用于RGB-D相机与机械臂标定及抓取任务。
摘要:
大型语言模型(LLMs)在包括代码生成和问题解决在内的各个领域展示了令人印象深刻的表现。然而,它们在机器人控制中的应用,特别是在需要精确操作、实时反馈和环境依赖执行的低级任务中,仍然有限。为了解决这一挑战,我们提出了闭环模块化代码合成框架。该框架利用预训练的LLM,无需任何任务特定的微调,执行模块化代码规划和生成,并在迭代执行生成的代码的同时插入调试探针以观察其行为。这种闭环结构促进了系统性的调试和优化,最终生成可执行的控制程序。我们将该框架应用于RGB-D相机和机械臂的标定,验证了其在真实世界环境中的有效性。此外,通过后续的抓取任务,我们不仅展示了标定的准确性,还展示了框架的潜在可扩展性。在两个任务中,该框架都实现了高执行准确性和自主性,说明了使用我们框架进行基于LLM的机器人控制的实用性和可扩展性。
英文摘要:
Large Language Models (LLMs) have demonstrated impressive performance across various domains, including code generation and problem solving. However, their application in robotic control, particularly in low-level tasks that require precise manipulation, real-time feedback, and environment-dependent execution, remains limited. To address this challenge, we propose the Closed-Loop Modular Code Synthesizer framework. This framework leverages a pre-trained LLM without any task-specific fine-tuning to perform modular code planning and generation, and iteratively executes the generated code while inserting debugging probes to observe its behavior. This closed-loop structure facilitates systematic debugging and refinement, ultimately producing executable control programs. We apply the proposed framework to the calibration of an RGB-D camera and a robotic arm, validating its effectiveness in real-world settings. Furthermore, through a subsequent pick-and-place task, we demonstrate not only the accuracy of the calibration but also the potential extensibility of the framework. Across both tasks, the framework achieved high execution accuracy and autonomy, illustrating the practicality and scalability of LLM-based robotic control using our framework.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。