DiffBench 与 DiffAgent:端到端的 LLM 驱动扩散加速代码生成
DiffBench Meets DiffAgent: End-to-End LLM-Driven Diffusion Acceleration Code Generation
AI总结:
本文提出DiffBench和DiffAgent,通过LLM驱动的闭环流程,实现端到端的扩散模型加速代码生成,显著提升生成策略的有效性。
摘要:
扩散模型在图像和视频生成中取得了显著成功。然而,其本质上多步骤的推理过程带来了巨大的计算开销,阻碍了实际应用。因此,加速扩散模型至关重要,但如何结合多种模型加速技术仍然是重大挑战。为了解决这一问题,我们引入了一个由大型语言模型(LLMs)驱动的框架,用于自动化加速代码的生成和评估。首先,我们提出了DiffBench,一个全面的基准,它在多样化的扩散架构、优化组合和部署场景上实现了三阶段的自动化评估流程。其次,我们提出了DiffAgent,一个能够为任意扩散模型生成最优加速策略和代码的代理。DiffAgent采用闭环工作流程,在规划组件和调试组件的迭代优化下,不断改进代码生成组件的输出,同时遗传算法从执行环境提取性能反馈,以指导后续的代码优化。我们详细解释了DiffBench的构建和DiffAgent的设计原则。大量实验表明,DiffBench能够全面评估生成的代码,而DiffAgent在生成有效的扩散加速策略方面显著优于现有LLMs。
英文摘要:
Diffusion models have achieved remarkable success in image and video generation. However, their inherently multiple step inference process imposes substantial computational overhead, hindering real-world deployment. Accelerating diffusion models is therefore essential, yet determining how to combine multiple model acceleration techniques remains a significant challenge. To address this issue, we introduce a framework driven by large language models (LLMs) for automated acceleration code generation and evaluation. First, we present DiffBench, a comprehensive benchmark that implements a three stage automated evaluation pipeline across diverse diffusion architectures, optimization combinations and deployment scenarios. Second, we propose DiffAgent, an agent that generates optimal acceleration strategies and codes for arbitrary diffusion models. DiffAgent employs a closed-loop workflow in which a planning component and a debugging component iteratively refine the output of a code generation component, while a genetic algorithm extracts performance feedback from the execution environment to guide subsequent code refinements. We provide a detailed explanation of the DiffBench construction and the design principles underlying DiffAgent. Extensive experiments show that DiffBench offers a thorough evaluation of generated codes and that DiffAgent significantly outperforms existing LLMs in producing effective diffusion acceleration strategies.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。