DiT-Head：使用 Diffusion Transformers 的高分辨率说话头像合成
DiT-Head: High-Resolution Talking Head Synthesis using Diffusion Transformers
AI总结：
提出基于 diffusion transformers 的 DiT-Head，以音频条件驱动扩散去噪，实现可跨身份泛化的高分辨率说话头像合成，并在视觉质量与唇形同步上达到可比现有方法的效果。
摘要：
我们提出了一种名为“DiT-Head”的新型说话头像合成流程，该流程基于 diffusion transformers，并使用音频作为条件来驱动扩散模型的去噪过程。我们的方法具有可扩展性，能够泛化到多种身份，同时生成高质量结果。我们对所提出的方法进行了训练和评估，并将其与现有的说话头像合成方法进行比较。结果表明，我们的模型在视觉质量和唇形同步准确性方面能够与这些方法竞争。我们的结果凸显了所提方法在虚拟助手、娱乐和教育等广泛应用中的潜力。有关结果的视频演示和我们的用户研究，请参阅补充材料。
英文摘要：
We propose a novel talking head synthesis pipeline called "DiT-Head", which is based on diffusion transformers and uses audio as a condition to drive the denoising process of a diffusion model. Our method is scalable and can generalise to multiple identities while producing high-quality results. We train and evaluate our proposed approach and compare it against existing methods of talking head synthesis. We show that our model can compete with these methods in terms of visual quality and lip-sync accuracy. Our results highlight the potential of our proposed approach to be used for a wide range of applications, including virtual assistants, entertainment, and education. For a video demonstration of the results and our user study, please refer to our supplementary material.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
