AVO: Agentic Variation Operators for Autonomous Evolutionary Search
NVIDIA
Terry Chen, Zhifan Ye, Bing Xu, Zihao Ye, Timmy Liu, Ali Hassani, Tianqi Chen, Andrew Kerr, Haicheng Wu, Yang Xu, Yu-Jung Chen, Hanfeng Chen, Aditya Kane, Ronny… 展开作者
Terry Chen, Zhifan Ye, Bing Xu, Zihao Ye, Timmy Liu, Ali Hassani, Tianqi Chen, Andrew Kerr, Haicheng Wu, Yang Xu, Yu-Jung Chen, Hanfeng Chen, Aditya Kane, Ronny Krashinsky, Ming-Yu Liu, Vinod Grover, Luis Ceze, Roger Bringmann, John Tran, Wei Liu, Fung Xie, Michael Lightstone, Humphrey Shi
Agentic Variation Operators (AVO) are a new family of evolutionary variation operators that replace the fixed mutation, crossover, and hand-designed heuristics of classical evolutionary search with autonomous coding agents. Rather than confining a language model to candidate generation within a prescribed pipeline, AVO instantiates variation as a self-directed agent loop that can consult the current lineage, a domain-specific knowledge base, and execution feedback to propose, repair, critique, and verify implementation edits. We evaluate AVO on attention, among the most aggressively optimized kernel targets in AI, on NVIDIA Blackwell (B200) GPUs. Over 7 days of continuous autonomous evolution on multi-head attention, AVO discovers kernels that outperform cuDNN by up to 3.5% and FlashAttention-4 by up to 10.5% across the evaluated configurations. The discovered optimizations transfer readily to grouped-query attention, requiring only 30 minutes of additional autonomous adaptation and yielding gains of up to 7.0% over cuDNN and 9.3% over FlashAttention-4. Together, these results show that agentic variation operators move beyond prior LLM-in-the-loop evolutionary pipelines by elevating the agent from candidate generator to variation operator, and can discover performance-critical micro-architectural optimizations that produce kernels surpassing state-of-the-art expert-engineered attention implementations on today's most advanced GPU hardware.
LLM2TEA: An Agentic AI Designer for Discovery with Generative Evolutionary Multitasking
1 College of Computing \& Data Science (CCDS), Nanyang Technological University (NTU), Singapore 2 Honda Research Institute Europe (HRI-EU), Offenbach am Main, Germany 3 Centre for Frontier AI Research (CFAR), Agency for Science, Technology
Evolutionary System Prompt Learning for Reinforcement Learning in LLMs
Department of Computer Science, University of Toronto(多伦多大学计算机科学系);Department of Statistics(统计学系);Data Science, Northwestern University(数据科学,西北大学);Bridgewater AIA Labs(布里奇沃特AIA实验室)
Deliberate Evolution: Agentic Reasoning for Sample-Efficient Symbolic Regression with LLMs
TMLR Group, Department of Computer Science, Hong Kong Baptist University(香港 Baptist 大学计算机科学系 TMLR 组);Beijing National Research Center for Information Science(北京信息科学国家研究中心);Technology (BNRist), Department of Automation, Tsinghua University, Beijing, P.R. China(技术(BNRist),自动化系,清华大学,北京,中华人民共和国);Lenovo Research(联想研究)