多模态学习遇见遗传编程：分析潜在空间优化中的对齐
Multi-Modal Learning meets Genetic Programming: Analyzing Alignment in Latent Space Optimization
AI总结：
本文研究SNIP方法在符号回归中多模态潜在空间优化的对齐效果，发现跨模态对齐在优化过程中未改善且过于粗糙，导致有效对齐引导的优化尚未实现。
摘要：
符号回归旨在从数据中发现数学表达式，传统上通过遗传编程对符号结构进行组合搜索来解决。潜在空间优化方法使用神经编码器将符号表达式映射到连续空间，将组合搜索转化为连续优化。受CLIP启发的对比预训练模型SNIP（Meidani等人，2024）通过引入多模态方法推进了潜在空间优化：在共享潜在空间中对齐符号和数值编码器以学习表型-基因型映射，从而在数值空间中进行优化以隐式指导符号搜索。然而，这依赖于细粒度的跨模态对齐，而类似CLIP模型的文献表明这种对齐通常是粗粒度的。在本文中，我们研究SNIP是否实现了其对符号回归进行有效双模态优化的承诺。我们的实验表明：（1）跨模态对齐在优化过程中并未改善，即使适应度增加；（2）SNIP学习的对齐过于粗糙，无法在符号空间中高效进行原则性搜索。这些发现揭示了尽管多模态潜在空间优化对符号回归具有巨大潜力，但有效的对齐引导优化在实践中仍未实现，突显了细粒度对齐作为未来工作的关键方向。
英文摘要：
Symbolic regression (SR) aims to discover mathematical expressions from data, a task traditionally tackled using Genetic Programming (GP) through combinatorial search over symbolic structures. Latent Space Optimization (LSO) methods use neural encoders to map symbolic expressions into continuous spaces, transforming the combinatorial search into continuous optimization. SNIP (Meidani et al., 2024), a contrastive pre-training model inspired by CLIP, advances LSO by introducing a multi-modal approach: aligning symbolic and numeric encoders in a shared latent space to learn the phenotype-genotype mapping, enabling optimization in the numeric space to implicitly guide symbolic search. However, this relies on fine-grained cross-modal alignment, whereas literature on similar models like CLIP reveals that such an alignment is typically coarse-grained. In this paper, we investigate whether SNIP delivers on its promise of effective bi-modal optimization for SR. Our experiments show that: (1) cross-modal alignment does not improve during optimization, even as fitness increases, and (2) the alignment learned by SNIP is too coarse to efficiently conduct principled search in the symbolic space. These findings reveal that while multi-modal LSO holds significant potential for SR, effective alignment-guided optimization remains unrealized in practice, highlighting fine-grained alignment as a critical direction for future work.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
