迈向连续运动-语言智能体：用于增量运动理解与生成的LoRA变体
Towards Continual Motion-Language Agents: LoRA Variants for Incremental Motion Understanding and Generation
AI总结：
针对运动-语言智能体在连续任务中学习新运动概念时灾难性遗忘的问题，提出基于LoRA的混合专家架构，通过自编码器路由器选择任务特定专家，无需任务标签，在HumanML3D基准上实现双向任务近零遗忘并保持高质量。
摘要：
运动-语言智能体必须拥有双向能力：既理解人类运动（运动到文本，M2T），又能从自然语言生成运动（文本到运动，T2M）。虽然基础模型在静态设置中取得了强劲性能，但在动态环境中运行的自主智能体必须持续融入新的运动概念——例如新颖的运动风格或专门的手势——而不会灾难性地遗忘先前习得的技能。我们研究了在顺序任务暴露下双向运动-语言学习中的稳定性-可塑性权衡。基于冻结的大语言模型骨干，我们引入了低秩适应（LoRA）变体以减轻任务间干扰。我们特别提出了混合专家架构，利用基于自编码器的路由器在推理时选择任务特定专家，因此无需任务标签。为了评估这些方法，我们通过运动描述的语义聚类，从HumanML3D建立了一个可重复的五任务基准。我们的实验结果表明，在M2T和T2M两个方向上均实现了接近零的遗忘，同时保持了高生成和描述质量。此外，我们展示了通过路由进行硬专家选择在质量指标上显著优于软专家混合，表明保持专家隔离对于在我们的持续学习设置中维持性能至关重要。最后，我们观察到令牌级准确率与下游生成质量之间可能出现分歧，这凸显了在未来关于终身运动-语言智能体的研究中需要更全面的评估协议。
英文摘要：
Motion-language agents must possess the bidirectional capability to both understand human movement (motion-to-text, M2T) and generate it from natural language (text-to-motion, T2M). While foundational models have achieved strong performance in static settings, autonomous agents operating in dynamic environments must continuously incorporate new motion concepts -- such as novel athletic styles or specialized gestures -- without catastrophic forgetting of previously acquired skills. We investigate the stability-plasticity trade-off in bidirectional motion-language learning under sequential task exposure. Building on a frozen large language model backbone, we introduce low-rank adaptation (LoRA) variants designed to mitigate inter-task interference. We specifically propose mixture-of-experts architectures that utilize an autoencoder-based router to select task-specific experts at inference time, so that no task-label is needed. To evaluate these methods, we establish a reproducible five-task benchmark derived from HumanML3D through semantic clustering of motion descriptions. Our experimental results demonstrate near-zero forgetting across both M2T and T2M directions while maintaining high generation and captioning quality. Furthermore, we show that hard expert selection via routing significantly outperforms soft expert blending in quality metrics, indicating that preserving expert isolation is critical for maintaining performance in our continual learning setting. Finally, we observe that a divergence between token-level accuracy and downstream generation quality may occur, highlighting the need for more comprehensive evaluation protocols in future research on lifelong motion-language agents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
