发表机构
CUHK-Shenzhen; University of British Columbia（香港中文大学（深圳）; 不列颠哥伦比亚大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对自动化核设计有效性与表达性的矛盾，提出Kernaut框架，将核设计视为开放式模型发现，通过构造契约保证有效性，利用质量-多样性档案和新颖性筛选发现可泛化、可解释的核，实验证明其优于元学习深度核并可由人类改进。
AI 中文摘要
核函数编码了广泛机器学习模型的归纳偏置，然而自动化核设计面临一个根本性困境。固定的基核与算子语法保证了有效性，但将搜索限制在可由这些构建块表达的结构内。相反，无限制的程序消除了这一限制，但不再保证有效性。在我们的压力测试中，22-58%的由大语言模型生成的核函数在随机输入上通过数值检查，但在不同尺度或维度下评估时失败。我们提出核自动研究（Kernaut），将核设计视为开放式模型发现。编码代理将核编写为程序，而构造契约确保每个被接受的核都是有效的。一个质量-多样性档案库保留具有不同行为的高性能核，新颖性筛选引导代理朝向功能上新的候选。我们的实验表明，所发现的核编码了可复用的归纳偏置，能泛化到未见任务。在保留的黑盒优化族上，一个发现的核优于在同一情节上训练的元学习深度核。此外，从十个酶动力学速率定律中发现的核在五个未见机制上实现了比调优的ARD和深度核基线更低的误差。所发现的核也是可解释的程序，人类研究者可以改进：一个人类改进版本进一步将保留的预测误差降低了5.7%，优化遗憾降低了7.8%。
英文摘要
Kernels encode the inductive bias of a wide range of machine learning models, yet automated kernel design faces a fundamental dilemma. A fixed grammar of base kernels and operators guarantees validity but limits the search to structures expressible by those building blocks. Conversely, unrestricted programs remove this limitation but no longer guarantee validity. In our stress tests, 22-58% of LLM-generated kernels that pass numerical checks on random inputs fail when evaluated at different scales or dimensions. We propose Kernel Autoresearch (Kernaut), which treats kernel design as open-ended model discovery. Coding agents write kernels as programs, while construction contracts ensure that every accepted kernel is valid. A quality-diversity archive retains high-performing kernels with distinct behaviors, and novelty screening steers agents toward functionally new candidates. Our experiments demonstrate that the discovered kernels encode reusable inductive biases that generalize to unseen tasks. On held-out black-box optimization families, a discovered kernel outperforms a meta-learned deep kernel trained on the same episodes. Furthermore, kernels discovered from ten enzyme-kinetic rate laws achieve lower error than tuned ARD and deep kernel baselines on five unseen mechanisms. The discovered kernels are also interpretable programs that human researchers can refine: a human-refined version of one further reduces the held-out predictive error by 5.7% and optimization regret by 7.8%.
