GACA-DiT：基于扩散的舞蹈到音乐生成，具有风格自适应节奏和上下文感知对齐
GACA-DiT: Diffusion-based Dance-to-Music Generation with Genre-Adaptive Rhythm and Context-Aware Alignment
AI总结：
GACA-DiT通过风格自适应节奏提取和上下文感知时间对齐模块，实现了舞蹈到音乐生成的节奏一致性和时间对齐，提升了生成音乐与舞蹈动作的同步精度。
摘要：
舞蹈到音乐（D2M）生成旨在自动编排与舞蹈动作在节奏和时间上对齐的音乐。现有方法通常依赖于粗粒度的节奏嵌入，如全局运动特征或二值化的关节基节奏值，这些方法丢弃了细粒度的运动线索，导致节奏对齐薄弱。此外，特征下采样引入的时间错配进一步阻碍了舞蹈与音乐之间的精确同步。为了解决这些问题，我们提出了GACA-DiT，一种基于扩散变换器的框架，包含两个新颖的模块，用于生成节奏一致且时间对齐的音乐。首先，一个风格自适应节奏提取模块结合多尺度时间小波分析和空间相位直方图，结合自适应关节加权，以捕捉细粒度、风格特定的节奏模式。其次，一个上下文感知的时间对齐模块利用可学习的上下文查询来对齐音乐潜在特征与相关的舞蹈节奏特征。在AIST++和TikTok数据集上的广泛实验表明，GACA-DiT在客观指标和人类评估中均优于现有最先进方法。项目页面：https://beria-moon.github.io/GACA-DiT/。
英文摘要：
Dance-to-music (D2M) generation aims to automatically compose music that is rhythmically and temporally aligned with dance movements. Existing methods typically rely on coarse rhythm embeddings, such as global motion features or binarized joint-based rhythm values, which discard fine-grained motion cues and result in weak rhythmic alignment. Moreover, temporal mismatches introduced by feature downsampling further hinder precise synchronization between dance and music. To address these problems, we propose \textbf{GACA-DiT}, a diffusion transformer-based framework with two novel modules for rhythmically consistent and temporally aligned music generation. First, a \textbf{genre-adaptive rhythm extraction} module combines multi-scale temporal wavelet analysis and spatial phase histograms with adaptive joint weighting to capture fine-grained, genre-specific rhythm patterns. Second, a \textbf{context-aware temporal alignment} module resolves temporal mismatches using learnable context queries to align music latents with relevant dance rhythm features. Extensive experiments on the AIST++ and TikTok datasets demonstrate that GACA-DiT outperforms state-of-the-art methods in both objective metrics and human evaluation. Project page: https://beria-moon.github.io/GACA-DiT/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
