ASPIRin: 动作空间投影用于全双工语音语言模型中的交互优化强化学习
ASPIRin: Action Space Projection for Interactivity-Optimized Reinforcement Learning in Full-Duplex Speech Language Models
AI总结：
ASPIRin通过动作空间投影和GRPO方法优化全双工语音语言模型的交互性能，提升语义连贯性并减少重复生成。
摘要：
端到端全双工语音语言模型（SLMs）需要精确的轮流交互以实现自然交互。然而，通过标准原始令牌强化学习（RL）优化时间动态会降低语义质量，导致严重的生成崩溃和重复。我们提出了ASPIRin，一种交互优化的RL框架，明确解耦说话时间和说话内容。通过动作空间投影，ASPIRin将文本词汇映射到粗粒度的二进制状态（主动说话 vs. 不活跃沉默）。通过应用基于规则的奖励的组相对策略优化（GRPO），它平衡了用户中断和响应延迟。实证评估显示，ASPIRin在轮流交互、回声处理和暂停处理方面优化了交互性。关键的是，将时间与令牌选择分离保持了语义连贯性，并将重复n-gram的比例比标准GRPO降低了超过50%，有效消除了退化重复。
英文摘要：
End-to-end full-duplex Speech Language Models (SLMs) require precise turn-taking for natural interaction. However, optimizing temporal dynamics via standard raw-token reinforcement learning (RL) degrades semantic quality, causing severe generative collapse and repetition. We propose ASPIRin, an interactivity-optimized RL framework that explicitly decouples when to speak from what to say. Using Action Space Projection, ASPIRin maps the text vocabulary into a coarse-grained binary state (active speech vs. inactive silence). By applying Group Relative Policy Optimization (GRPO) with rule-based rewards, it balances user interruption and response latency. Empirical evaluations show ASPIRin optimizes interactivity across turn-taking, backchanneling, and pause handling. Crucially, isolating timing from token selection preserves semantic coherence and reduces the portion of duplicate n-grams by over 50% compared to standard GRPO, effectively eliminating degenerative repetition.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
