During both pretraining and fine-tuning, Large Language Models (\textbf{LLMs}) are trained on trillions of tokens of text of widely varying quality. Both phases of training typically involve heuristically filtering out ``low-quality'' or \textit{noisy} training samples, yet little is known quantitatively about how the type or intensity of noise affects downstream performance. In this work, we study how noise in chain of thought (\textbf{CoT}) impacts task performance in the highly-controlled setting of algorithmically solvable tasks. First, we develop the Traced Integer (\textbf{TInt}) framework to generate highly customizable noised execution traces for any arithmetic function on lists of integers. We then define two types of noise: \textit{static} noise, a local form of noise which is applied after the CoT trace is computed, and \textit{dynamic} noise, a global form of noise which propagates errors in the trace as it is computed. We then evaluate the test performance of pretrained models both prompted and fine-tuned on noised datasets with varying levels of dataset contamination and intensity. We find fine-tuned models are extremely robust to high levels of static noise but struggle significantly more with lower levels of dynamic noise. In contrast, few-shot prompted models appear more sensitive to even static noise. We conclude with a discussion of how our findings impact noise filtering best-practices, in particular emphasizing the importance of removing samples containing destructive dynamic noise with global errors.
OCEAN: Offline Chain-of-thought Evaluation and Alignment in Large Language Models
UC San Diego(加州大学圣迭戈分校);The University of New South Wales(新南威尔士大学);East China Normal University(华东师范大学);Adobe Research(奥多比研究中心);CSIRO’s Data61(联邦科学与工业研究组织数据61部门)
Thought Purity: A Defense Framework For Chain-of-Thought Attack
Huzhou University(湖州大学);Zhejiang Key Laboratory of Intelligent Education Technology and Application(浙江智能教育技术与应用重点实验室);University of Science and Technology of China(中国科学技术大学);Alibaba Group(阿里巴巴集团);Zhejiang University of Technology(浙江工业大学)
A Theory of Learning with Autoregressive Chain of Thought
Toyota Technological Institute at Chicago(丰田技术研究所(芝加哥));Weizmann Institute of Science(魏兹曼科学研究院);Microsoft Research, NYC(微软研究院(纽约));University of Pennsylvania(宾夕法尼亚大学);Yale University(耶鲁大学)
Enhancing Granular Sentiment Classification with Chain-of-Thought Prompting in Large Language Models
John F Kennedy High school Fremont, USA(约翰·F·肯尼迪高中弗里蒙特分校);Queensland Academy for Science Mathematics and Technology(昆士兰科学数学与技术学院);Research Spark Hub Inc(Research Spark Hub公司)
Generating Chain-of-Thoughts with a Pairwise-Comparison Approach to Searching for the Most Promising Intermediate Thought
Center for Advanced Intelligence Project, RIKEN(理化学研究所先进智能项目中心);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);Graduate School of Frontier Sciences, The University of Tokyo(东京大学前沿科学研究生院)
Training Multimodal Large Reasoning Models Needs Better Thoughts: A Three-Stage Framework for Long Chain-of-Thought Synthesis and Selection
School of Computer Science and Engineering(计算机科学与工程学院);Key Laboratory of Computer Network and Information Integration (SEU), Ministry of Education(计算机网络与信息集成重点实验室(SEU))
Iteration Head: A Mechanistic Study of Chain-of-Thought
FAIR, Meta AI(FAIR,Meta人工智能公司);Datashape, INRIA(Datashape,法国国家信息与自动化研究所);Courant University and Center for Data Science, NYU(纽约大学柯朗数学科学研究所与数据科学中心);Center for Data Science, NYU(纽约大学数据科学中心);NYU(纽约大学);Courant University(柯朗大学)
Compositional Reasoning with Transformers, RNNs, and Chain of Thought
Courant Institute of Mathematical Sciences(Courant数学科学研究所);New York University(纽约大学);Center for Data Science(数据科学中心);Center for Computational Mathematics(计算数学中心);Flatiron Institute(Flatiron研究所)
Think Consistently, Reason Efficiently: Energy-Based Calibration for Implicit Chain-of-Thought
University of Oxford(牛津大学);Tsinghua University(清华大学);Tencent(腾讯);Southern University of Science and Technology(南方科技大学);National University of Singapore(新加坡国立大学)