TinyAudio：面向低资源部署的紧凑高效文本到音频生成

中文摘要 AI 辅助

文本到音频（TTA）生成在生成质量和指令跟随方面已取得快速进展。然而，代表性系统通常需要约十亿参数，限制了在资源受限设备上的部署。本文介绍了TinyAudio，一种基于流匹配的紧凑TTA模型，专为低资源部署设计。其核心是TA-DiT，一个35M的单流流匹配Transformer。TinyAudio还包含TA-CLAP，一个32M的音频对齐文本编码器，以及TA-VAE，其20M解码器从压缩潜变量重建44.1 kHz音频。TinyAudio总参数仅为87M，比代表性的十亿参数流水线减少了超过90%，峰值GPU内存使用为0.48 GB。TinyAudio在AudioCaps和TTA-Bench上实现了具有竞争力的生成质量。我们进一步引入了TinyAudio-MF，一种MeanFlow加速模型，能够在四核CPU配额下实现实时生成。我们的结果展示了低资源TTA部署中质量与占用之间的实用权衡。

英文摘要

Text-to-audio (TTA) generation has advanced rapidly in generation quality and instruction following. However, representative systems often require around a billion parameters, limiting deployment on resource-constrained devices. This paper introduces TinyAudio, a compact flow-matching-based TTA model for low-resource deployment. At its core, TinyAudio uses TA-DiT, a 35M single-stream flow-matching Transformer. TinyAudio also includes TA-CLAP, a 32M audio-aligned text encoder, and TA-VAE, whose 20M decoder reconstructs 44.1 kHz audio from compressed latents. TinyAudio has only 87M parameters in total, over 90% fewer than representative billion-parameter pipelines, and uses 0.48 GB peak GPU memory. TinyAudio achieves competitive generation quality on AudioCaps and TTA-Bench. We further introduce TinyAudio-MF, a MeanFlow-accelerated model that enables real-time generation with a four-core CPU quota. Our results demonstrate a practical quality-footprint trade-off for low-resource TTA deployment.

发表机构

• X-LANCE Lab, Shanghai Jiao Tong University（上海交通大学X-LANCE实验室）
• Shanghai Innovation Institute（上海创新研究院）
• SJTU Paris Elite Institute of Technology, Shanghai Jiao Tong University（上海交通大学巴黎卓越工程师学院）
• Nanyang Technological University（南洋理工大学）

机构由 AI 辅助整理，请以论文原文为准。