arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.08706cs.LGcs.AI
收藏

在潜在程序空间中搜索

Searching Latent Program Spaces

University of Amsterdam(阿姆斯特丹大学)

Matthew V Macfarlane, Clement Bonnet

AI总结:

本文提出LPN,一种在测试时直接构建程序搜索能力的神经网络,通过学习隐含程序空间实现高效适应新任务。

摘要:

通用智能需要能够高效获取新技能并超越训练分布的系统。尽管程序合成方法具有强大的泛化能力,但它们由于较大的组合空间而面临扩展问题,这种组合空间很快使它们变得不切实际,需要人工生成的DSL或预训练的先验来缩小搜索空间。另一方面,深度学习方法已取得显著成功,但它们缺乏结构化的测试时适应能力,并且依赖于大量的随机采样或昂贵的梯度更新来进行微调。在本工作中,我们提出了潜在程序网络(LPN),这是一种新的架构,它在神经模型中直接构建测试时搜索。LPN通过隐含程序的潜在空间学习,即通过神经映射输入到输出,从而在测试时使用梯度进行搜索。LPN结合了符号方法的适应性和神经方法的可扩展性。它在测试时通过紧凑的潜在空间进行搜索,并绕过了需要预定义领域特定语言的需要。在一系列编程通过示例任务上,LPN在测试时学习和测试时训练方法上要么表现更优,要么表现相当。在ARC-AGI基准测试上,我们证明了LPN既能学习紧凑的程序空间,又能在测试时通过该空间进行搜索以适应新任务。当启用测试时搜索时,LPN在超出分布任务上的表现翻倍。

英文摘要:

General intelligence requires systems that acquire new skills efficiently and generalize beyond their training distributions. Although program synthesis approaches have strong generalization power, they face scaling issues due to the large combinatorial spaces that quickly render them impractical, requiring human-generated DSLs or pre-trained priors to narrow this search space. On the other hand, deep learning methods have had high successes, but they lack structured test-time adaptation and rely on heavy stochastic sampling or expensive gradient updates for fine-tuning. In this work, we propose the Latent Program Network (LPN), a novel architecture that builds in test-time search directly into neural models. LPN learns a latent space of implicit programs -- neurally mapping inputs to outputs -- through which it can search using gradients at test time. LPN combines the adaptability of symbolic approaches and the scalability of neural methods. It searches through a compact latent space at test time and bypasses the need for pre-defined domain-specific languages. On a range of programming-by-examples tasks, LPN either outperforms or matches performance compared to in-context learning and test-time training methods. Tested on the ARC-AGI benchmark, we demonstrate that LPN can both learn a compact program space and search through it at test time to adapt to novel tasks. LPN doubles its performance on out-of-distribution tasks when test-time search is switched on.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑