Language Models are Symbolic Learners in Arithmetic
Department of Computer Science(计算机科学系);Rice University(里士满大学);College of Computing(计算学院);Georgia Institute of Technology(佐治亚理工学院);Duke University(杜克大学)
Chunyuan Deng, Zhiqi Li, Roy Xie, Ruidi Chang, Hanjie Chen
The prevailing question in LM performing arithmetic is whether these models learn to truly compute or if they simply master superficial pattern matching. In this paper, we argues for the latter, presenting evidence that LMs act as greedy symbolic learners, prioritizing the simplest possible shortcuts to fit the stats of dataset to solve arithmetic tasks. To investigate this, we introduce subgroup induction, a practical framework adapted from Solomonoff Induction (SI), one of the most powerful universal predictors. Our framework analyzes arithmetic problems by breaking them down into subgroups-minimal mappings between a few input digits and a single output digit. Our primary metric, subgroup quality, measures the viability of these shortcuts. Experiments reveal a distinct U-shaped accuracy pattern in multi-digit multiplication: LMs quickly master the first and last output digits while struggling with those in the middle. We demonstrate this U-shape is not coincidental; it perfectly mirrors the quality of the simplest possible subgroups, those requiring the fewest input tokens. This alignment suggests a core learning mechanism: LMs first learn easy, low-token shortcuts and only incorporate more complex, multi-token patterns as training progresses. They do not learn the algorithm of multiplication but rather a hierarchy of increasingly complex symbol-to-symbol mappings. Ultimately, our findings suggest that the path to arithmetic mastery for LMs is not paved with algorithms, but with a cascade of simple, hierarchically-learned symbolic shortcuts.
The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision
MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);IIIS, Tsinghua University(清华大学交叉信息研究院);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);Deepmind(深度思考);MIT BCS(麻省理工学院脑与认知科学系);CBMM(脑、心智与机器中心)
Physics Symbolic Learner for Discovering Ground-Motion Models Via NGA-West2 Database
Beijing University of Technology(北京工业大学);Institute of Geophysics, China Earthquake Administration(中国地震局地球物理研究所);Institute of Geomechanics, Chinese Academy of Geological Sciences(中国地质科学院地质力学研究所)
Adaptive Protection for Evolutionary Feature Construction in Symbolic Regression with Application to Credit Classification
Centre for Data Science and Artificial Intelligence(数据科学与人工智能中心);School of Engineering and Computer Science(工程与计算机科学学院);Victoria University of Wellington(惠灵顿维多利亚大学);Business School, Sichuan University(四川大学商学院);Michigan State University(密歇根州立大学)