Language models struggle with handling numerical data and performing arithmetic operations. We hypothesize that this limitation can be partially attributed to non-intuitive textual numbers representation. When a digit is read or generated by a causal language model it does not know its place value (e.g. thousands vs. hundreds) until the entire number is processed. To address this issue, we propose a simple adjustment to how numbers are represented by including the count of digits before each number. For instance, instead of "42", we suggest using "{2:42}" as the new format. This approach, which we term NumeroLogic, offers an added advantage in number generation by serving as a Chain of Thought (CoT). By requiring the model to consider the number of digits first, it enhances the reasoning process before generating the actual number. We use arithmetic tasks to demonstrate the effectiveness of the NumeroLogic formatting. We further demonstrate NumeroLogic applicability to general natural language modeling, improving language understanding performance in the MMLU benchmark.
StreamingThinker: Large Language Models Can Think While Reading
Shanghai Jiao Tong University(上海交通大学);Eastern Institute of Technology, Ningbo(宁波东部技术研究院);Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Institute of Digital Twin(宁波空间智能与数字衍生关键实验室,数字孪生研究院);Hong Kong Polytechnic University(香港理工大学);Munich Center for Machine Learning, LMU(慕尼黑机器学习中心,莱茵-慕尼黑大学)
ReGuLaR: Variational Latent Reasoning Guided by Rendered Chain-of-Thought
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学 Gallagher人工智能学院);Beijing Key Laboratory of Research on Large Models(北京大型模型研究关键实验室);Engineering Research Center of Next-Generation Intelligent Search(下一代智能搜索工程研究中心)
EndoCoT: Scaling Endogenous Chain-of-Thought Reasoning in Diffusion Models
Shanghai AI Laboratory(上海人工智能实验室);Xi’an Jiaotong University(西安交通大学);University of Science and Technology of China(中国科学技术大学);Shanghai Jiaotong University(上海交通大学);Fudan University(复旦大学);The Chinese University of Hong Kong(香港中文大学)
Great Truths are Always Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);School of Information, Renmin University of China(中国人民大学信息学院);Beijing Academy of Artificial Intelligence(北京智源人工智能研究院)
R$^2$ec: Towards Large Recommender Models with Reasoning
The Hong Kong Polytechnic University(香港理工大学);National University of Singapore(新加坡国立大学);University of Science and Technology of China(中国科学技术大学);Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳))
Large language models can learn and generalize steganographic chain-of-thought under process supervision
Mentorship for Alignment Research Students (MARS)(对齐研究 mentorship 项目);University College London(伦敦大学学院);Queen Mary University of London(伦敦女王学院);ML Alignment & Theory Scholars (MATS)(对齐与理论学者);Meridian Impact, Cambridge(剑桥 Meridian Impact);Universidad Carlos III de Madrid(马德里卡洛斯三世大学);Geodesic Research and University of Cambridge(Geodesic Research 和剑桥大学)
Chatting with Images for Introspective Visual Thinking
NLPR, MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Nanjing University(南京大学)
Studying the Soupability of Documents in State Space Models
Department of Computer Science, University of California San Diego, San Diego, CA, USA(加州大学圣地亚哥分校计算机科学系);University of Illinois Urbana-Champaign, Champaign, IL, USA(伊利诺伊大学厄巴纳-香槟分校)