LongAV-Compass:面向分钟级音视频生成在T2AV、I2AV和V2AV上的统一评估
LongAV-Compass: Towards Unified Evaluation of Minute-Scale Audio-Visual Generation Across T2AV, I2AV, and V2AV
AI总结:
针对现有评估协议局限于短片段的问题,提出LongAV-Compass基准,通过284个测试用例和统一评估框架,系统评估分钟级音视频生成在文本、图像、视频条件下的质量、一致性和对齐。
摘要:
音视频生成正从短片段快速发展到分钟级内容,而现有评估协议仍主要局限于短片段设置。现有基准主要关注5-10秒的文本条件生成,很少支持跨文本、图像和视频条件模态的统一评估。此外,它们对身份一致性、叙事连贯性和音视频对齐在长时间跨度上的退化提供的洞察有限。为弥补这一差距,我们引入了LongAV-Compass,一个用于分钟级音视频生成的系统基准。LongAV-Compass包含284个精选测试用例,涵盖文本到音视频(T2AV)、图像到音视频(I2AV)和视频到音视频(V2AV),按应用场景和生成复杂度组织。该基准结合了基于分类法的基准构建和统一评估框架,该框架集成了MLLM辅助评估与互补的感知和多模态指标,包括DINO-v2、ArcFace、CLIP和ImageBind。该框架评估超过20个细粒度维度,涵盖片段内质量、跨片段一致性、全局叙事连贯性、语义对齐和音视频同步。通过对11个代表性模型的实验以及人类对齐验证,LongAV-Compass提供了一个诊断测试平台,用于分析当前系统在跨不同输入模态维持连贯、语义对齐和时间一致的分钟级音视频生成方面的局限性。
英文摘要:
Audio-visual generation is rapidly advancing from short clips to minute-long content, while existing evaluation protocols remain largely confined to short-form settings. Existing benchmarks primarily focus on 5--10 second text-conditioned generation and rarely support unified evaluation across text, image, and video conditioning modalities. Moreover, they provide limited insight into how identity consistency, narrative coherence, and audio-visual alignment degrade over extended temporal horizons. To bridge this gap, we introduce LongAV-Compass, a systematic benchmark for minute-long audio-visual generation. LongAV-Compass contains 284 curated test cases spanning text-to-audio-video (T2AV), image-to-audio-video (I2AV), and video-to-audio-video (V2AV), organized by application scenario and generation complexity. The benchmark combines taxonomy-guided benchmark construction with a unified evaluation framework that integrates MLLM-assisted assessment with complementary perceptual and multimodal metrics, including DINO-v2, ArcFace, CLIP, and ImageBind. The framework evaluates more than 20 fine-grained dimensions covering within-segment quality, cross-segment consistency, global narrative coherence, semantic alignment, and audio-visual synchronization. Through experiments on 11 representative models together with human-alignment validation, LongAV-Compass provides a diagnostic testbed for analyzing the limitations of current systems in sustaining coherent, semantically aligned, and temporally consistent minute-scale audio-visual generation across diverse input modalities.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。