面向昇腾 Triton 内核生成的感知压缩强化学习
Compaction-Aware Reinforcement Learning For Ascend Triton Kernel Generation
AI总结:
针对 LLM 智能体生成高性能内核的上下文窗口受限与奖励稀疏问题,提出 ATRL 方法,结合昇腾专属知识库,在 NPUKernelBench 等基准上实现优于基线的内核生成效果。
摘要:
基于大语言模型(LLM)的智能体自动生成高性能内核有望大幅提升开发效率,但仍存在两个障碍:其一,增加与执行环境的多轮交互对优化生成质量至关重要,而持续的上下文累积会迅速超出模型的有效上下文窗口;其二,通过 GRPO 训练长周期任务时,由于依赖终端结果评估,会固有地产生稀疏奖励,严重阻碍步骤级的信用分配。为解决这些障碍,我们提出 ATRL(Ascend Triton Reinforcement Learning,昇腾 Triton 强化学习),这是一种后训练方法,将 GRPO 优化为感知压缩的分段目标,用于长周期内核生成。我们还设计了轻量级智能体框架,提供级联编译、正确性和性能分析反馈,并在智能体强化学习训练与评估中注入昇腾专属知识库,以弥补基础模型缺失的平台专业知识。在 NPUKernelBench 和 KernelBench 上的评估(均在昇腾硬件上执行)显示,ATRL 在正确性和加速比上均持续优于包括 GRPO 在内的所有基线。
英文摘要:
Automatically generating high-performance kernels with LLM-based agents promises substantial gains in development efficiency, but two obstacles remain. First, while increasing multi-turn interaction with execution environments is crucial for refining generation quality, continuous context accumulation rapidly exceeds the model's effective context window. Second, training long-horizon tasks via GRPO inherently incurs sparse rewards due to its reliance on terminal outcome evaluation, severely hindering step-level credit assignment; To address these obstacles, we propose ATRL (Ascend Triton Reinforcement Learning), a post-training method that refines GRPO into a compaction-aware, segment-level objective for long-horizon kernel generation. We also design a lightweight agent harness providing cascaded compilation, correctness, and profiling feedback, alongside an Ascend-specific knowledge base injected during agentic RL training and evaluation to compensate for the base model's missing platform expertise. Evaluations on NPUKernelBench and KernelBench, both executed on Ascend hardware, show that ATRL consistently surpasses all baselines, including GRPO, in both correctness and speedup.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。