arXiv 2606.104922026-06-09cs.CV

PathRelax: 并行路径松弛推测雅可比解码加速自回归文本到图像生成

PathRelax: Parallel-Path Relaxed Speculative Jacobi Decoding for Accelerating Auto-Regressive Text-to-Image Generation

College of Software Engineering, Southeast University（东南大学软件工程学院）；School of Computer Science and Engineering, Southeast University（东南大学计算机科学与工程学院）；School of Cyber Science and Engineering, Southeast University（东南大学网络空间安全学院）；School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算机与信息系统学院）

Haodong Lei, Hongsong Wang, Bingxuan Dai, Pan Zhou

备注：: 10 pages, 5 figures

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

针对自回归文本到图像模型因长序列导致推理慢的问题，提出并行路径交叉松弛推测雅可比解码框架，通过多序列草稿树结构扩展搜索空间并利用跨路径语义相似性提高接受率，实现3.95-4.18倍加速。

摘要：

自回归文本到图像模型对高分辨率图像生成的需求日益增长，导致令牌序列延长，显著增加了计算成本和推理时间。然而，现有的加速自回归文本到图像模型的最先进方法依赖于链式结构的草稿令牌序列，导致草稿令牌搜索效率低下且接受长度有限。为了解决这一问题，我们提出了并行路径交叉松弛推测雅可比解码（PathSpec），一种通过多序列草稿树结构提升效率的新框架。我们的并行路径推测雅可比解码（PathExplore）扩展了令牌搜索空间，在不牺牲图像质量的情况下实现了更高的加速比。此外，我们引入了跨路径松弛验证（PathRelax），利用序列间的语义相似性进一步提高令牌接受率。在Parti-Prompts、MSCOCO2017和T2ICompBench数据集上的评估表明，我们的方法分别实现了4.14倍、3.95倍和4.18倍的加速比。值得注意的是，PathExplore在没有任何松弛采样的条件下，在加速比上优于GSD和LANTERN等松弛采样方法。此外，PathRelax的松弛机制可以与其他松弛技术无缝集成，实现进一步加速，为实时文本到图像生成提供了高效解决方案。我们的代码可在该https URL获取。

英文摘要：

The growing need for high-resolution image generation in autoregressive text-to-image models has resulted in extended token sequences, significantly increasing computational costs and inference times. However, existing state-of-the-art methods for accelerating autoregressive text-to-image models rely on chain-structured draft token sequences, leading to inefficient draft token search and limited acceptance lengths. To address this, we propose parallel-path cross-relaxed speculative Jacobi decoding (\textbf{PathSpec}), a novel framework that enhances efficiency through a multi-sequence draft tree structure. Our parallel-path speculative Jacobi decoding (\textbf{PathExplore}) expands the token search space, achieving a higher speedup ratio without sacrificing image quality. Additionally, we introduce cross-path relaxed verification (\textbf{PathRelax}) that exploits semantic similarities across sequences to further boost token acceptance rates. Evaluated on the Parti-Prompts, MSCOCO2017, and T2ICompBench datasets, our method achieves a speedup ratio of 4.14 $\times$, 3.95$\times$, and 4.18$\times$, respectively. Remarkably, PathExplore, without any relaxed sampling, outperforms relaxed sampling methods in the speedup ratio, such as GSD and LANTERN. Moreover, PathRelax's relaxation mechanism can be seamlessly integrated with other relaxation techniques, enabling further acceleration and providing an efficient solution for real-time text-to-image generation. Our code is available at https://github.com/Haodong-Lei-Ray/PathSpec.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。