ACE-Step 1.5:推动开源音乐生成的边界
ACE-Step 1.5: Pushing the Boundaries of Open-Source Music Generation
AI总结:
ACE-Step 1.5通过高效开源模型实现商业级音乐生成,结合内在强化学习与混合架构,支持多语言创作与风格控制。
摘要:
我们介绍了ACE-Step v1.5,一个高效且开源的音乐基础模型,它将商业级的生成能力带入消费者硬件。在常用的评估指标上,ACE-Step v1.5在质量上超越了大多数商业音乐模型,同时保持了极高的速度——在A100上每首完整歌曲不到2秒,在RTX 3090上不到10秒。该模型可以在少于4GB VRAM的本地设备上运行,并支持轻量级的个性化:用户只需用几首歌曲训练一个LoRA即可捕捉到自己的风格。其核心是一种新颖的混合架构,其中语言模型(LM)作为全能的规划器:它将简单的用户查询转化为全面的歌曲蓝图——从短循环到10分钟的创作——同时通过链式推理合成元数据、歌词和描述,以指导扩散变压器(DiT)。独特的是,这种对齐是通过内在强化学习实现的,仅依赖于模型内部机制,从而消除了外部奖励模型或人类偏好的偏差。除了标准合成外,ACE-Step v1.5还统一了精确的风格控制与多样的编辑能力——如封面生成、重绘和人声转背景音乐——同时在50多种语言上严格遵循提示。这为强大的工具铺平了道路,这些工具可以无缝地融入音乐艺术家、制作人和内容创作者的创意工作流程中。代码、模型权重和演示可在:https://ace-step.github.io/ace-step-v1.5.github.io/上获得。
英文摘要:
We present ACE-Step v1.5, a highly efficient open-source music foundation model that brings commercial-grade generation to consumer hardware. On commonly used evaluation metrics, ACE-Step v1.5 achieves quality beyond most commercial music models while remaining extremely fast -- under 2 seconds per full song on an A100 and under 10 seconds on an RTX 3090. The model runs locally with less than 4GB of VRAM, and supports lightweight personalization: users can train a LoRA from just a few songs to capture their own style. At its core lies a novel hybrid architecture where the Language Model (LM) functions as an omni-capable planner: it transforms simple user queries into comprehensive song blueprints -- scaling from short loops to 10-minute compositions -- while synthesizing metadata, lyrics, and captions via Chain-of-Thought to guide the Diffusion Transformer (DiT). Uniquely, this alignment is achieved through intrinsic reinforcement learning relying solely on the model's internal mechanisms, thereby eliminating the biases inherent in external reward models or human preferences. Beyond standard synthesis, ACE-Step v1.5 unifies precise stylistic control with versatile editing capabilities -- such as cover generation, repainting, and vocal-to-BGM conversion -- while maintaining strict adherence to prompts across 50+ languages. This paves the way for powerful tools that seamlessly integrate into the creative workflows of music artists, producers, and content creators. The code, the model weights and the demo are available at: https://ace-step.github.io/ace-step-v1.5.github.io/
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。