SalsaAgent: 一种用于交互式舞蹈生成的多模态具身语言模型
SalsaAgent: A multimodal embodied language model for interactive dance generation
AI总结：
提出SalsaAgent语言模型，通过非语言运动令牌传递和两阶段令牌到扩散管道，生成与人类领舞者及音乐背景交互的全身萨尔萨舞蹈动作。
摘要：
人形机器人之间的交互涉及双向和非语言反应性、协调与同步。为了构建具有社会意识的机器人和交互式虚拟代理，我们提出了SalsaAgent，一种语言模型，能够生成表达性的全身萨尔萨舞蹈动作，以响应人类领舞者并配合背景音乐。我们将交互形式化为非语言运动令牌传递，扩展了大语言模型（LLM）的词汇表，以处理离散运动令牌、成对关系令牌和音频。我们的贡献包括：用于全身和运动关系的新令牌、使用自动推导的骨架动力学文本描述进行令牌对齐的LLM微调，以及两阶段令牌到扩散管道。主观和客观评估表明，我们的方法在运动质量、音乐与伙伴协调以及一致的双人空间行为方面具有有效性，显著优于基线方法。
英文摘要：
Embodied interaction with humanoids depends on bidirectional nonverbal reactivity, coordination, and synchrony to convey cues and move with a partner. For socially interactive embodied agents, reactive motion generation requires expressive full-body motion that remains contextually appropriate while maintaining spatial and temporal synchrony. We present SalsaAgent, a language model that generates expressive, full-body salsa follower motions in reaction to a human leader and music. We formulate partner interaction as nonverbal token passing, extending the vocabulary of a large language model (LLM) to process discrete motion tokens, pairwise relation tokens, and audio tokens. Our method introduces full-body and pairwise-relation tokenizers, aligns language and motion tokens with automatically derived text descriptions of skeleton dynamics, and applies a two-stage token-to-diffusion pipeline. Subjective and objective evaluations show improved motion quality, two-person spatial coordination, and music and partner coordination relative to prior baselines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
