LongCat-Video-Avatar 1.5 技术报告
LongCat-Video-Avatar 1.5 Technical Report
AI总结:
本文提出 LongCat-Video-Avatar 1.5,一个通过升级音频编码器、优化训练策略、数据筛选和RLHF训练实现高精度唇同步、全身时间稳定性和长视频生成的开放框架,在多个基准测试中达到或超越商业系统性能。
摘要:
尽管音频驱动视频生成取得了进展,但实现商业级稳定性仍具挑战。我们提出 LongCat-Video-Avatar 1.5,一个升级的开源框架,优先考虑系统工程和生产就绪性而非架构新颖性。通过将音频编码器升级为 Whisper Large 并精心扩展训练配方,v1.5 实现了精确的唇同步、全身时间稳定性和严格身份一致性的鲁棒长视频生成。通过严格的数据筛选和 RLHF 训练,该模型能轻松泛化到风格化领域(如动漫和动物),并原生处理复杂现实条件(如多人交互和物体操作)。此外,针对工业部署的实际需求,我们采用高级步进蒸馏将推理加速至最优的 8 NFE,在服务效率与视觉保真度之间实现了良好权衡。通过在超过 500 个多样化测试案例的综合基准上进行的广泛定量指标和严格人工评估,验证了我们方法的优越性。结果表明,v1.5 在人类相似度评分和专家级质量评估中,与领先的闭源系统(如 HeyGen、OmniHuman 1.5、Kling Avatar 2.0)相比,达到了具有竞争力或更优的性能。通过开源发布,LongCat-Video-Avatar 1.5 缩小了学术研究原型与商业级部署之间的差距。
英文摘要:
Despite advances in audio-driven video generation, achieving commercial-grade stability remains challenging. We present LongCat-Video-Avatar 1.5, an upgraded open-source framework prioritizing systematic engineering and production-readiness over architectural novelty. By upgrading the audio encoder to Whisper Large and meticulously scaling our training recipes, v1.5 achieves accurate lip-synchronization, full-body temporal stability, and robust long-video generation with strict identity consistency. Through rigorous data curation and RLHF Training, the model readily generalizes to stylized domains such as anime and animals, and natively handles complex real-world conditions, such as multi-person interactions and object handling. Furthermore, addressing the practical demands of industrial deployment, we employ advanced step distillation to accelerate inference to an optimal 8 NFE, achieving a favorable trade-off between serving efficiency and visual fidelity. The superiority of our approach is validated through extensive quantitative metrics and a rigorous human evaluation conducted on a comprehensive benchmark of over 500 diverse test cases. Results show that v1.5 achieves competitive or superior performance compared to leading closed-source systems (e.g., HeyGen, OmniHuman 1.5, Kling Avatar 2.0) across human-likeness ratings and expert-level quality assessments on our benchmark. With its open-source release, LongCat-Video-Avatar 1.5 narrows the gap between academic research prototypes and commercial-grade deployment.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。