Instruct2Act: 从人类指令到动作序列和执行的机器人操作网络
Instruct2Act: From Human Instruction to Actions Sequencing and Execution via Robot Action Network for Robotic Manipulation
AI总结：
Instruct2Act通过机器人操作网络实现从人类指令到动作序列的解析与执行，达到91.5%的子动作预测准确率和90%的任务成功率。
摘要：
机器人在真实世界环境中往往难以遵循自由形式的人类指令，这主要是由于计算和感知限制。我们通过一个轻量级、完全在设备上的管道来解决这一差距，将自然语言命令转换为可靠的操纵。我们的方法分为两个阶段：（i）指令到动作模块（Instruct2Act），一个紧凑的BiLSTM加多头注意力自动编码器，将指令解析为有序的原子动作序列（例如，到达、抓取、移动、放置）；（ii）机器人操作网络（RAN），它使用动态自适应轨迹径向网络（DATRN）和基于视觉的环境分析器（YOLOv8）来为每个子动作生成精确的控制轨迹。整个系统运行在一个适度的系统上，没有云服务。在我们定制的专有数据集中，Instruct2Act在保持小足迹的同时，达到91.5%的子动作预测准确率。在四个任务（拾取-放置、拾取-倒酒、擦拭、拾取-给予）上的真实机器人评估中，总体成功率达到90%；子动作推理在<3.8秒内完成，端到端执行时间在30-60秒之间，取决于任务复杂性。这些结果表明，细粒度的指令到动作解析，结合DATRN基于的轨迹生成和视觉引导的定位，为在资源受限、单摄像头环境下实现确定性和实时操纵提供了可行的路径。
英文摘要：
Robots often struggle to follow free-form human instructions in real-world settings due to computational and sensing limitations. We address this gap with a lightweight, fully on-device pipeline that converts natural-language commands into reliable manipulation. Our approach has two stages: (i) the instruction to actions module (Instruct2Act), a compact BiLSTM with a multi-head-attention autoencoder that parses an instruction into an ordered sequence of atomic actions (e.g., reach, grasp, move, place); and (ii) the robot action network (RAN), which uses the dynamic adaptive trajectory radial network (DATRN) together with a vision-based environment analyzer (YOLOv8) to generate precise control trajectories for each sub-action. The entire system runs on a modest system with no cloud services. On our custom proprietary dataset, Instruct2Act attains 91.5% sub-actions prediction accuracy while retaining a small footprint. Real-robot evaluations across four tasks (pick-place, pick-pour, wipe, and pick-give) yield an overall 90% success; sub-action inference completes in < 3.8s, with end-to-end executions in 30-60s depending on task complexity. These results demonstrate that fine-grained instruction-to-action parsing, coupled with DATRN-based trajectory generation and vision-guided grounding, provides a practical path to deterministic, real-time manipulation in resource-constrained, single-camera settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
