基于上下文的符号回归用于改进鲁棒性的Kolmogorov-Arnold网络
In-Context Symbolic Regression for Robustness-Improved Kolmogorov-Arnold Networks
AI总结：
本文提出两种改进KANs符号提取的方法，通过上下文感知的贪心策略和门控匹配追踪，提升鲁棒性并减少预测误差，实验显示在多个任务中达到99.8%的中位数MSE降低。
摘要：
基于上下文的符号回归旨在通过简洁的解析表达式替代黑箱预测器，以提升科学机器学习中的可解释性和可验证性。Kolmogorov-Arnold网络（KANs）因其每个相邻单元之间的连接（称为“边”）由可学习的单变量函数参数化，而非常适合这一目标。然而，实际中符号提取是一个瓶颈：传统的KAN到符号方法是将操作符拟合到每个学习的边函数上，这使得离散选择对初始化敏感且非凸参数拟合问题，且忽略了局部替换通过整个网络的相互作用。我们研究了KANs中基于上下文的符号回归用于操作符提取，并提出了两种互补的实现。贪心基于上下文的符号回归（GSR）通过端到端损失改进来选择边替换，进行简短的微调。门控匹配追踪（GMP）通过训练一个可微的门控操作符层来放大这种基于上下文的选择，该层在每个边后放置一个操作符库，通过稀疏门控；在收敛后，门控被离散化（可选地随后进行短时间基于上下文的贪心细化）。我们通过单因素-at-a-time（OFAT）超参数扫描量化鲁棒性，并评估了恢复公式预测误差和定性一致性。在多个实验中，基于上下文的贪心符号回归实现了高达99.8%的中位数OFAT测试MSE降低。
英文摘要：
Symbolic regression aims to replace black-box predictors with concise analytical expressions that can be inspected and validated in scientific machine learning. Kolmogorov-Arnold Networks (KANs) are well suited to this goal because each connection between adjacent units (an "edge") is parametrised by a learnable univariate function that can, in principle, be replaced by a symbolic operator. In practice, however, symbolic extraction is a bottleneck: the standard KAN-to-symbol approach fits operators to each learned edge function in isolation, making the discrete choice sensitive to initialisation and non-convex parameter fitting, and ignoring how local substitutions interact through the full network. We study in-context symbolic regression for operator extraction in KANs, and present two complementary instantiations. Greedy in-context Symbolic Regression (GSR) performs greedy, in-context selection by choosing edge replacements according to end-to-end loss improvement after brief fine-tuning. Gated Matching Pursuit (GMP) amortises this in-context selection by training a differentiable gated operator layer that places an operator library behind sparse gates on each edge; after convergence, gates are discretised (optionally followed by a short in-context greedy refinement pass). We quantify robustness via one-factor-at-a-time (OFAT) hyper-parameter sweeps and assess both predictive error and qualitative consistency of recovered formulas. Across several experiments, greedy in-context symbolic regression achieves up to 99.8% reduction in median OFAT test MSE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
