从捷径学习到离散神经插入排序
From Shortcut Learning to Discrete Neural Insertion Sort
浏览论文内容
中文总结 AI 辅助
针对神经算法推理中模型走捷径而非真正执行算法的问题，本文提出离散神经插入排序，通过离散化与状态监督，在长度16训练下实现长度64和128上100%准确率，并揭示全局状态监督的必要性。
中文摘要 AI 辅助
神经算法推理旨在训练神经网络遵循已知算法，并泛化到训练时未见过的输入规模。然而，正确的最终输出和中间监督并不一定表明模型遵循了预期的执行过程。我们以插入排序为例研究这一问题。对CLRS30基线NAR的分析表明，提示目标被弱优化，提示准确率仍然较低。此外，许多中间表示在参考插入排序执行终止之前就已经可以被解码为有序序列，这表明模型学习了一条通往最终输出的捷径。基于这些发现，我们引入了离散神经插入排序。我们的模型将序列表示为一条链，将标量交换与控制状态转换分离，并在每个处理器步骤后将节点表示投影回离散状态。当仅在长度为16的序列上训练时，模型在长度为64和128的序列上实现了100%的有序序列准确率。然而，消融实验表明，仅靠离散化和图结构是不够的：如果没有对全局内循环状态的额外监督，模型即使在训练长度上也会失败。我们的结果表明，离散执行可以支持强大的长度泛化，同时也凸显了学习忠实算法执行所需的特定于问题的归纳偏置。
英文摘要
Neural algorithmic reasoning aims to train neural networks to follow known algorithms and generalize beyond the input sizes seen during training. However, correct final outputs and intermediate supervision do not necessarily show that a model follows the intended execution. We study this problem using insertion sort. Our analysis of the CLRS30 baseline NAR shows that the hint objective is weakly optimized and that hint accuracy remains low. Moreover, many intermediate representations can already be decoded into sorted sequences before the reference insertion-sort execution terminates, suggesting that the model learns a shortcut to the final output. Motivated by these findings, we introduce Discrete Neural Insertion Sort. Our model represents the sequence as a chain, separates scalar exchanges from control-state transitions, and projects node representations back to discrete states after every processor step. When trained only on sequences of length 16, the model achieves $100\%$ sorted-sequence accuracy on sequences of length 64 and 128. However, an ablation shows that discretization and graph structure alone are insufficient: without additional supervision of the global inner-loop state, the model fails even at the training length. Our results show that discrete execution can support strong length generalization, while also highlighting the problem-specific inductive bias required to learn a faithful algorithmic execution.
