Kandinsky 6.0 Video：用于同步视频和音频生成的基础模型

Kandinsky 6.0 Video: Foundation Models for Synchronized Video and Audio Generation

Team Kandinsky, Julia Agafonova, Bulat Akhmatov, Mikhail Aksyutin, Grigorii Alekseenko, Anastasia Aliaskina, Olga Androsova, Vladimir Arkhipkin, Anna Averchenkova, Alexander Belykh, Serafima Bocharova, Sofiya Bogakovskaya, Anton Bukashkin, Mark Bulygin, Kirill Buzygin, Irina Cheremnykh, Kirill Chernyshev, Mikhail Chernyshov, Vladimir Chernyy, David Chikovani, Georgy Daniltsev, Denis Dimitrov, Anna Dmitrienko, Vladimir Dokholyan, Sergey Emelyanov, Dmitry Ermilov, Georgii Fedorov, Polina Gavrilova, Nikolai Gerasimenko, Aleksandr Gordeev, Andrey Inozemtsev, Andrei Ivaniuta, Alexander Ivanov, Mikhail Karaev, Anastasiia Kargapoltseva, Ivan Kirillov, Nikita Kiselev, Valeria Kobenko, Yury Kolabushin, Denis Koposov, Anatoly Korobov, Vladimir Korviakov, Kirill Kozlov, Denis Krzhivokolskiy, Konstantin Kuklev, Alexander Kunitsyn, Sergey Kuzin, Vladislav Lakhtionov, Alexey Letunovskiy, Maxim Litvinov, Alexander Lyulkov, Georgy Makarov, Kirill Malakhov, Egor Malykh, Mikhail Mamaev, Dmitrii Mikhailov, Polina Mikhailova, Ivan Mikheev, Elizaveta Muromtseva, Nikolai Nazarkin, Tatiana Nikulina, Lev Novitskiy, Stanislav Onuchin, Nikita Osterov, Denis Parkhomenko, Anatoliy Parpara, Vladimir Polovnikov, Konstantin Reznikov, Azat Saginbaev, Nikita Samsonov, Alexander Sentsov, Nikita Shaimov, Artem Sherstyuk, Andrey Shutkin, Egor Silvestrov, Bulat Suleimanov, Matvey Suprunov, Sergey Taranov, Irina Tolstykh, Tatiana Trofimuk, Ilya Trushkin, Aleksandra Tsybina, Olga Varlashina, Viacheslav Vasilev, Ilya Vasiliev, Eugeny Vilisov, Sergey Yakubson, Konstantin Zakharov

arXiv 2610.05608首次发表：2026-10-06更新：2026-10-06

发表机构

Kandinsky Lab（康定斯基实验室）

机构由 AI 辅助整理，请以论文原文为准。

AI 中文总结

本文提出 Kandinsky 6.0 Video 基础扩散模型系列，通过双流 CrossDiT 架构和连续预训练策略，实现同步音频-视频生成，并在人工评估中优于前代模型，且开源发布。

AI 中文摘要

我们提出了 Kandinsky 6.0 Video，一个用于同步文本到音频-视频生成的基础扩散模型系列，包括 Kandinsky 6.0 Video Lite（3B 参数）和 Kandinsky 6.0 Video Pro（29B 参数）。两个模型都能生成 5 秒的视频片段，并带有同步的 44 kHz 音频，包括唇形同步，支持文本到音频-视频（T2AV）和图像到音频-视频（I2AV）两种模式；内置的超分辨率模型将输出分辨率提升至全高清（1920×1080）。在 Kandinsky 5.0 的视频生成能力基础上，Kandinsky 6.0 Video 采用了双流 CrossDiT 架构，通过双向交叉注意力连接预训练的视频流和新训练的音频流，以实现时间和语义上的对齐。我们的连续预训练策略首先在大规模音频语料库上从头训练音频流，然后在配对的音频-视频数据上联合训练两个流，同时保持单模态保真度；预训练之后是监督微调、基于强化学习的后训练和蒸馏。在并排人工评估中，Kandinsky 6.0 Video Pro 明显优于其前身 Kandinsky 5.0 Video Pro，并与领先的音频-视频生成模型保持竞争力，尤其是在语音质量方面。为了加速多媒体生成领域的开放研究和部署，我们在 MIT 许可下发布了代码、模型检查点和 diffusers 集成。

英文摘要

We present Kandinsky 6.0 Video, a family of foundation diffusion models for synchronized text-to-audio-video generation, comprising Kandinsky 6.0 Video Lite (3B parameters) and Kandinsky 6.0 Video Pro (29B parameters). Both models generate 5-second video clips with synchronized 44 kHz audio, including lip-sync, in text-to-audio-video (T2AV) and image-to-audio-video (I2AV) modes; a built-in super-resolution model raises the output resolution to Full-HD (1920$\times$1080). Building on the video generation capabilities of Kandinsky 5.0, Kandinsky 6.0 Video employs a dual-stream CrossDiT architecture that connects a pretrained video stream and a newly trained audio stream through bidirectional cross-attention for temporal and semantic alignment. Our continuous pretraining strategy first trains the audio stream from scratch on large-scale audio corpora and then trains both streams jointly on paired audio-video data while preserving unimodal fidelity; pretraining is followed by supervised fine-tuning, reinforcement-learning-based post-training, and distillation. In side-by-side human evaluation, Kandinsky 6.0 Video Pro clearly outperforms its predecessor, Kandinsky 5.0 Video Pro, and remains competitive with leading audio-video generation models, particularly in speech quality. To accelerate open research and deployment in multimedia generation, we release the code, model checkpoints, and diffusers integration under the MIT license.

CommentsTechnical report on the open-source T2AV model. GitHub: https://github.com/kandinskylab/kandinsky-6

论文原文

arXiv 摘要页 · PDF 原文 · HTML 原文