Hi-Singers：用于表现力音频驱动歌唱头部合成的高质量综合数据集
Hi-Singers: A Comprehensive High-Quality Dataset for Expressive Audio-Driven Singing Head Synthesis
浏览论文内容
中文总结 AI 辅助
针对歌唱头部合成中语音模型面临的领域差距，提出首个大规模高质量野外数据集Hi-Singers，含29,608个片段约170小时，经严格过滤和基准评估，显著提升视觉真实感、口型同步和节奏动态。
中文摘要 AI 辅助
音频驱动数字人生成的最先进模型在说话头部合成中已取得照片级逼真效果。然而，将这些模型扩展到歌唱头部合成仍具挑战性，因为存在显著的领域差距：歌唱需要更夸张的表情、生动的下颌开合以及精确的节奏同步。当前主要基于语音数据集训练的模型，常面临“节奏漂移”和动态受限的问题。为解决此问题，我们引入了Hi-Singers，这是首个专为歌唱头部合成定制的大规模、高质量、野外视频数据集。Hi-Singers经过严格的自动和人工过滤流程，确保严格的主题一致性、高分辨率渲染和稳定的运动，最终包含29,608个视频片段，总计约170小时。我们进一步建立了一个在语言和音乐风格上均衡的专用评估基准。跨多种架构（包括3D系数和基于扩散的模型）的大量实验表明，Hi-Singers在所有维度上持续且显著地提升了性能。具体而言，它使模型能够实现更优的视觉真实感、增强的口型同步一致性以及更精确的节奏动态，有效弥合了领域差距，并为歌唱合成任务树立了新的性能标准。该数据集可在以下https URL获取。
英文摘要
State-of-the-art models for audio-driven digital human generation have achieved photo-realistic results in talking-head synthesis. However, extending these models to singing-head synthesis remains challenging due to a significant Domain Gap: singing requires more exaggerated expressions, vivid jaw openings, and precise rhythmic synchronization. Current models, primarily trained on speech datasets, often struggle with "rhythmic drift" and constrained dynamics. To address this, we introduce Hi-Singers, the first large-scale, high-quality, in-the-wild video dataset specifically tailored for singing head synthesis. Hi-Singers undergoes a rigorous automated and manual filtering pipeline, ensuring strict thematic adherence, high-resolution rendering, and stable motion, resulting in 29,608 video segments totaling approximately 170 hours. We further establish a dedicated evaluation benchmark balanced across linguistic and musical styles. Extensive experiments across diverse architectures, including 3D-coefficient and diffusion-based models, demonstrate that Hi-Singers consistently and significantly improves performance across all dimensions. Specifically, it enables models to achieve superior visual realism, enhanced lip-sync consistency, and more precise rhythmic dynamics, effectively bridging the domain gap and setting a new performance standard for the singing synthesis task. The dataset is available at https://huggingface.co/datasets/CharlesZhang-USTC/Hi-Singers
发表机构
- University of Science and Technology of China（中国科学技术大学）
机构由 AI 辅助整理，请以论文原文为准。
