arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2604.03473cs.CLcs.AI
收藏

为不确定性量化方法的自动化设计应用进化搜索

Evolutionary Search for Automated Design of Uncertainty Quantification Methods

AIRI;Skoltech(斯科尔科沃科学技术学院);Amazon(亚马逊);EPFL(瑞士联邦理工学院洛桑);MIPT(莫斯科物理技术学院)

Mikhail Seleznyov, Daniil Korbut, Viktor Moskvoretskii, Oleg Somov, Alexander Panchenko, Elena Tutubalina

AI总结:

本文通过LLM驱动的进化搜索自动发现无监督的不确定性量化方法,其在原子声明验证任务中优于手动设计的基线方法,且在分布外泛化方面表现稳健。

摘要:

大型语言模型的不确定性量化(UQ)方法大多依赖领域知识和启发式方法手动设计,限制了其可扩展性和通用性。我们应用LLM驱动的进化搜索自动发现以Python程序表示的无监督UQ方法。在原子声明验证任务中,我们的进化方法优于强大的手动设计基线,实现了9个数据集上的最大6.7%的相对ROC-AUC提升,同时在分布外泛化方面表现稳健。定性分析揭示了不同LLM采用不同的进化策略:Claude模型一致设计高特征计数的线性估计器,而Gpt-oss-120B则倾向于更简单且更具解释性的位置加权方案。令人惊讶的是,只有Sonnet 4.5和Opus 4.5能够利用增加的方法复杂性来提高性能--Opus 4.6相对于其前身出现了意外的退化。总体而言,我们的结果表明,LLM驱动的进化搜索是自动化、可解释的幻觉检测器设计的一种有前途的范式。

英文摘要:

Uncertainty quantification (UQ) methods for large language models are predominantly designed by hand based on domain knowledge and heuristics, limiting their scalability and generality. We apply LLM-powered evolutionary search to automatically discover unsupervised UQ methods represented as Python programs. On the task of atomic claim verification, our evolved methods outperform strong manually-designed baselines, achieving up to 6.7% relative ROC-AUC improvement across 9 datasets while generalizing robustly out-of-distribution. Qualitative analysis reveals that different LLMs employ qualitatively distinct evolutionary strategies: Claude models consistently design high-feature-count linear estimators, while Gpt-oss-120B gravitates toward simpler and more interpretable positional weighting schemes. Surprisingly, only Sonnet 4.5 and Opus 4.5 reliably leverage increased method complexity to improve performance -- Opus 4.6 shows an unexpected regression relative to its predecessor. Overall, our results indicate that LLM-powered evolutionary search is a promising paradigm for automated, interpretable hallucination detector design.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑