TA-V2A：文本辅助的视频到音频生成
TA-V2A: Textually Assisted Video-to-Audio Generation
AI总结：
针对视频到音频生成中帧特征易丢失序列上下文的问题，提出融合语言、音频与视频特征并由大语言模型和文本调制引导的 TA-V2A，以提升语义表达、时间对齐及生成质量。
摘要：
随着人工智能生成内容（AIGC）持续发展，视频到音频（V2A）生成已成为一个关键领域，在多媒体编辑、增强现实和自动化内容创作中具有广阔应用前景。尽管 Transformer 和 Diffusion 模型推动了音频生成的进步，但从视频中提取精确语义信息仍是一项重大挑战，因为当前模型往往仅依赖基于帧的特征，从而丢失序列上下文。为解决这一问题，我们提出 TA-V2A，该方法融合语言、音频和视频特征，以改善潜在空间中的语义表示。通过引入大型语言模型增强视频理解，我们的方法利用文本引导丰富语义表达。我们基于 diffusion 模型的系统采用自动化文本调制来提升推理质量和效率，并通过文本引导界面提供个性化控制。这种融合在增强语义表达的同时确保时间对齐，从而实现更准确、连贯的视频到音频生成。
英文摘要：
As artificial intelligence-generated content (AIGC) continues to evolve, video-to-audio (V2A) generation has emerged as a key area with promising applications in multimedia editing, augmented reality, and automated content creation. While Transformer and Diffusion models have advanced audio generation, a significant challenge persists in extracting precise semantic information from videos, as current models often lose sequential context by relying solely on frame-based features. To address this, we present TA-V2A, a method that integrates language, audio, and video features to improve semantic representation in latent space. By incorporating large language models for enhanced video comprehension, our approach leverages text guidance to enrich semantic expression. Our diffusion model-based system utilizes automated text modulation to enhance inference quality and efficiency, providing personalized control through text-guided interfaces. This integration enhances semantic expression while ensuring temporal alignment, leading to more accurate and coherent video-to-audio generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
