音频生成建模的统一视角：潜在表示与建模策略
A Unifying Perspective on Audio Generative Modeling: Latent Representations and Modeling Strategies
浏览论文内容
中文总结 AI 辅助
本文围绕音频生成建模中表示与分布建模的关联，提出基于依赖范围和条件歧义性的评估设计框架，揭示RVQ、AudioLM等系统的特性及不同建模策略的差异。
中文摘要 AI 辅助
每个音频生成系统都会做出两个相互关联的决策：生成何种表示，以及如何对其分布进行建模。本文围绕这种关联对音频生成建模进行组织。对于表示设计，我们通过四个目标（表示负担、失真、经验可建模性和流式兼容性）比较离散、连续和混合潜在变量。对于分布建模，我们不将潜在变量的难度视为固有标量，而是使用两个诊断维度：依赖范围（有用上下文延伸的距离）和条件歧义性（条件化后仍存在的不确定性程度）。这些维度细化了常见的语义-声学直觉：具有长依赖范围的变量应获得全局建模能力；条件歧义的细节可委托给局部或迭代生成器。将此视角应用于代表性系统后，结果显示：RVQ的残差顺序提供了有序的容量但非有序的语义；AudioLM的语义-声学级联是该边界的一种显式放置，而非通用模板；自回归、迭代细化和混合设计的主要区别在于它们在依赖范围与关键路径生成成本之间的权衡方式。离散与连续潜在变量的区分描述了输出接口，而依赖范围、条件歧义性和流式性决定了该接口应如何建模。我们未对单个系统进行分类，而是提供了一个用于比较表示-模型对的评估与设计框架。
英文摘要
Every audio generative system makes two coupled decisions: what representation to generate, and how to model its distribution. This paper organizes audio generative modeling around this coupling. For representation design, we compare discrete, continuous, and hybrid latents through four objectives: representation burden, distortion, empirical modelability, and streaming compatibility. For distribution modeling, rather than treating a latent's difficulty as an intrinsic scalar, we use two diagnostic dimensions: dependency horizon, how far useful context extends, and conditional ambiguity, how much uncertainty remains after conditioning. These dimensions refine the common semantic-versus-acoustic intuition: variables with a long dependency horizon should receive global modeling capacity. Conditionally ambiguous detail may be delegated to a local or iterative generator. Applied to representative systems, this view shows that RVQ's residual order gives ordered capacity but not ordered semantics, that AudioLM's semantic-versus-acoustic cascade is one explicit placement of this boundary rather than a universal template, and that autoregression, iterative refinement, and hybrid designs differ chiefly in how they trade dependency horizon against critical-path generation cost. The distinction between discrete and continuous latents describes the output interface; dependency horizon, conditional ambiguity, and streaming determine how that interface should be modeled. Rather than cataloguing individual systems, we provide an evaluation and design framework for comparing representation-model pairs.
