从LLM到行动：潜在代码作为分层机器人控制中的桥梁
From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control
AI总结：
针对分层机器人控制中语言接口的局限性，提出可学习潜在代码作为桥梁（LCB），连接LLM与低层策略，支持灵活通信与端到端微调，在Language Table和Calvin基准上优于纯语言基线。
摘要：
机器人学的分层控制长期以来一直受到需要有一个定义良好的接口层来连接高层任务规划器和低层策略的困扰。随着LLM的出现，语言正逐渐成为一种有前景的接口层。然而，这存在若干局限性。并非所有任务都能分解为易于用自然语言表达的步骤（例如，执行舞蹈套路）。此外，由于领域偏移和灾难性遗忘，在具身数据上进行端到端微调变得具有挑战性。我们引入了我们的方法——可学习潜在代码作为桥梁（LCB）——作为一种替代架构来克服这些局限性。LCB使用一个可学习的潜在代码作为LLM和低层策略之间的桥梁。这使得LLM能够灵活地在任务计划中传达目标，而不完全受语言限制的约束。此外，它使得端到端微调成为可能，而不会破坏预训练期间学到的词标记嵌入空间。通过在Language Table和Calvin这两个常见的基于语言的具身智能体基准上的实验，我们发现LCB在需要推理和多步行为的任务上优于利用纯语言作为接口层的基线方法（包括使用GPT-4V的基线）。
英文摘要：
Hierarchical control for robotics has long been plagued by the need to have a well defined interface layer to communicate between high-level task planners and low-level policies. With the advent of LLMs, language has been emerging as a prospective interface layer. However, this has several limitations. Not all tasks can be decomposed into steps that are easily expressible in natural language (e.g. performing a dance routine). Further, it makes end-to-end finetuning on embodied data challenging due to domain shift and catastrophic forgetting. We introduce our method -- Learnable Latent Codes as Bridges (LCB) -- as an alternate architecture to overcome these limitations. \method~uses a learnable latent code to act as a bridge between LLMs and low-level policies. This enables LLMs to flexibly communicate goals in the task plan without being entirely constrained by language limitations. Additionally, it enables end-to-end finetuning without destroying the embedding space of word tokens learned during pre-training. Through experiments on Language Table and Calvin, two common language based benchmarks for embodied agents, we find that \method~outperforms baselines (including those w/ GPT-4V) that leverage pure language as the interface layer on tasks that require reasoning and multi-step behaviors.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
