MIRA：面向具身伴侣的实时全双工人机交互
MIRA: Real-Time Full-Duplex Human-Robot Interaction for Embodied Companions
- International Digital Economy Academy（国际数字经济学院）
- Astribot
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
MIRA提出统一全双工框架，通过双时间尺度策略CORTEX和预测多于承诺的滑动窗口，实现流式语音驱动的实时、可中断且安全的具身伴侣交互，并在Astribot S1上验证了性能。
AI中文摘要：
实时的具身伴侣交互要求机器人从流式语音中推断用户意图，生成及时的响应，并执行富有表现力且可中断的动作。现有系统通常将对话编排与手势生成解耦，依赖从完整音频进行的离线动作生成。这种分离留下了这样一个问题：部署的机器人如何在增量输入和不确定的说话轮次边界下，动态地同步响应内容、韵律时序和物理安全。我们提出了MIRA，一个用于全双工具身伴侣交互的统一框架。给定流式用户语音、对话历史和声音情感，MIRA同时预测响应文本和显式的具身线索。离散的社交行为（例如，倾听、问候）被映射到经过验证的机器人轨迹，而开放式说话则与流式、伴随语音的动作配对。这种生成式动作由一个“预测多于承诺”的滑动窗口控制，该窗口为动作连续性提供时间前瞻，同时将物理承诺限制在一个短的、可取消的前缀内。关键的是，我们设计了CORTEX，一个双时间尺度的交互策略，它管理低延迟流式和深思熟虑的轮次决策，并由一个机器人侧执行层支持，该层在控制频率上强制执行物理安全约束。我们在Astribot S1人形机器人上部署了MIRA。定量评估表明，相对于最先进的动作生成基线，音频-动作对齐具有竞争力，而真实机器人部署测量则表征了流式响应性和中断处理。
英文摘要：
% !TEX root = ../main.tex Real-time embodied companion interaction requires a robot to infer user intent from streaming speech, generate timely responses, and execute expressive, interruptible motions. Existing systems typically decouple dialogue orchestration from gesture synthesis, relying on offline motion generation from complete audio. This separation leaves open how a deployed robot can dynamically synchronize response content, prosodic timing, and physical safety under incremental inputs and uncertain turn boundaries. We present MIRA, a unified framework for real-time full-duplex embodied companion interaction. Given streaming user speech, dialogue history, and vocal affect, MIRA predicts both the response text and an explicit embodiment cue that routes the response to the appropriate physical behavior. Discrete social behaviors (\eg listening and greeting) are mapped to validated robot trajectories, while speaking responses are accompanied by streaming, generative co-speech motion. For co-speech motion generation, we propose ROSCO, a prefix-conditioned diffusion model for streaming audio-to-joint motion generation. We further design RHPC, an inference scheme that maintains a sufficiently long temporal context for motion prediction while bounding physical commitment to a short, interruptible prefix. At the interaction level, we design CORTEX, a dual-timescale interaction policy that combines low-latency barge-in preemption and streaming response generation with deliberative turn decisions, backed by a robot-side execution layer that enforces physical safety constraints during execution. MIRA is deployed on an Astribot S1 humanoid robot. Quantitative evaluations demonstrate competitive audio-motion alignment relative to state-of-the-art motion-generation baselines, while real-robot deployment measurements characterize streaming responsiveness and interruption handling.
