BAD：面向文本生成动作的双向自回归扩散模型
BAD: Bidirectional Auto-regressive Diffusion for Text-to-Motion Generation
AI总结：
针对自回归模型难捕捉双向模式、掩码模型易削弱序列依赖的问题，提出融合二者优势的双向自回归扩散模型BAD，通过排列破坏技术兼顾序列与双向关系，在文本生成动作任务上表现更优，提供新的序列预训练策略。
摘要：
自回归模型通过施加因果约束在序列依赖建模方面表现出色，但由于其单向性，难以捕捉复杂的双向模式。相比之下，基于掩码的模型利用双向上下文，能够实现更丰富的依赖建模。然而，这类模型在预测时通常假设令牌相互独立，这会削弱序列依赖的建模效果。此外，通过掩码或吸收对序列进行破坏可能会引入不自然的失真，使学习过程复杂化。为解决这些问题，我们提出了双向自回归扩散模型（Bidirectional Autoregressive Diffusion，BAD），这是一种融合自回归模型与基于掩码的生成模型优势的新方法。BAD采用基于排列的破坏技术，在保留自然序列结构的同时，通过随机排序施加因果依赖，从而有效捕捉序列关系与双向关系。综合实验表明，BAD在文本生成动作任务上的表现优于自回归模型和基于掩码的模型，为序列建模提供了一种新的预训练策略。BAD的代码库可在https://github.com/RohollahHS/BAD获取。
英文摘要：
Autoregressive models excel in modeling sequential dependencies by enforcing causal constraints, yet they struggle to capture complex bidirectional patterns due to their unidirectional nature. In contrast, mask-based models leverage bidirectional context, enabling richer dependency modeling. However, they often assume token independence during prediction, which undermines the modeling of sequential dependencies. Additionally, the corruption of sequences through masking or absorption can introduce unnatural distortions, complicating the learning process. To address these issues, we propose Bidirectional Autoregressive Diffusion (BAD), a novel approach that unifies the strengths of autoregressive and mask-based generative models. BAD utilizes a permutation-based corruption technique that preserves the natural sequence structure while enforcing causal dependencies through randomized ordering, enabling the effective capture of both sequential and bidirectional relationships. Comprehensive experiments show that BAD outperforms autoregressive and mask-based models in text-to-motion generation, suggesting a novel pre-training strategy for sequence modeling. The codebase for BAD is available on https://github.com/RohollahHS/BAD.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
