arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.18603cs.LGcs.AIcs.CL
收藏

MMSR:符号回归是一种多模态信息融合任务

MMSR: Symbolic Regression is a Multi-Modal Information Fusion Task

Institute of Semiconductor, Chinese Academy of Sciences(中国科学院半导体研究所);University of Chinese Academy of Sciences(中国科学院大学);AnnLab(安实验室)

Yanjie Li, Jingyi Liu, Weijun Li, Lina Yu, Min Wu, Wenqiang Li, Meilan Hao, Su Wei, Yusong Deng

AI总结:

提出MMSR,将符号回归视为多模态信息融合任务,通过同时训练对比学习损失与其他损失实现模态对齐与融合,在SRBench等数据集上取得最先进结果。

摘要:

数学公式是人类数千年来探索自然规律智慧的结晶。用简洁的数学公式描述复杂的自然规律是科学家们不懈的追求,也是人工智能面临的巨大挑战。这一领域被称为符号回归(Symbolic Regression, SR)。符号回归最初被表述为一个组合优化问题,并采用遗传编程(Genetic Programming, GP)和强化学习算法来求解。然而,GP对超参数敏感,且这两类算法效率低下。为解决这一问题,研究者将数据到表达式的映射视为一个翻译问题,并引入了相应的大规模预训练模型。然而,数据与表达式骨架之间并不像两种语言那样具有非常清晰的单词对应关系,它们更像两种模态(例如图像和文本)。因此,在本文中,我们提出了MMSR。我们将SR问题作为纯多模态问题求解,并在训练过程中引入对比学习以实现模态对齐,从而促进后续的模态特征融合。值得注意的是,为了更好地促进模态特征融合,我们采用了同时训练对比学习损失和其他损失的策略,这只需要一步训练,而不是先训练对比学习损失再训练其他损失。因为我们的实验证明,同时训练可以使特征提取模块和特征融合模块更好地磨合。实验结果表明,与多个大规模预训练基线相比,MMSR在包括SRBench在内的多个主流数据集上取得了最先进的结果。我们的代码已在https://github.com/1716757342/MMSR开源。

英文摘要:

Mathematical formulas are the crystallization of human wisdom in exploring the laws of nature for thousands of years. Describing the complex laws of nature with a concise mathematical formula is a constant pursuit of scientists and a great challenge for artificial intelligence. This field is called symbolic regression (SR). Symbolic regression was originally formulated as a combinatorial optimization problem, and Genetic Programming (GP) and Reinforcement Learning algorithms were used to solve it. However, GP is sensitive to hyperparameters, and these two types of algorithms are inefficient. To solve this problem, researchers treat the mapping from data to expressions as a translation problem. And the corresponding large-scale pre-trained model is introduced. However, the data and expression skeletons do not have very clear word correspondences as the two languages do. Instead, they are more like two modalities (e.g., image and text). Therefore, in this paper, we proposed MMSR. The SR problem is solved as a pure multi-modal problem, and contrastive learning is also introduced in the training process for modal alignment to facilitate later modal feature fusion. It is worth noting that to better promote the modal feature fusion, we adopt the strategy of training contrastive learning loss and other losses at the same time, which only needs one-step training, instead of training contrastive learning loss first and then training other losses. Because our experiments prove training together can make the feature extraction module and feature fusion module wearing-in better. Experimental results show that compared with multiple large-scale pre-training baselines, MMSR achieves the most advanced results on multiple mainstream datasets including SRBench. Our code is open source at https://github.com/1716757342/MMSR

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑