This paper elucidates that current state-of-the-art Large Language Models (LLMs) are fundamentally incapable of making decisions or developing "thoughts" within the feature space due to their architectural constraints. We establish a definition of "thought" that encompasses traditional understandings of that term and adapt it for application to LLMs. We demonstrate that the architectural design and language modeling training methodology of contemporary LLMs inherently preclude them from engaging in genuine thought processes. Our primary focus is on this theoretical realization rather than practical insights derived from experimental data. Finally, we propose solutions to enable thought processes within the feature space and discuss the broader implications of these architectural modifications.
Toward Reasonable Parrots: Why Large Language Models Should Argue with Us by Design
University of Liverpool(利兹大学);University of Edinburgh(爱丁堡大学);University of Groningen(格罗宁根大学);Centrum Wiskunde & Informatica(数学与信息学研究中心);Airbus Central R&T(空客中央研发与技术部);University of Kassel(卡塞尔大学);University of Passau(帕绍大学);Universidad de Católica de la Santísima de Concepción(圣地亚哥圣母大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Dundee(邓迪大学);Leibniz University Hannover(汉诺威莱布尼茨大学)
Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do
The Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所复杂系统认知与决策智能实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Why Did Apple Fall: Evaluating Curiosity in Large Language Models
Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University(上海数据科学 key laboratory,计算机科学学院,复旦大学);Huawei Large Model Data Technology Lab(华为大模型数据技术实验室);Cornell University(康奈尔大学)
Why Ranking Anomaly Detection Algorithms Isn't as Reliable as You May Think
Carnegie Mellon University(卡内基梅隆大学);TU Dortmund University(多特蒙德工业大学);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习与人工智能研究所);iits Consulting GmbH(iits咨询有限公司);Fraunhofer Institute for Material Flow and Logistics IML(弗劳恩霍夫物流与材料流动研究所IML)
CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
LUMIA Lab(LUMIA实验室);School of Artificial Intelligence(人工智能学院);Shanghai Jiao Tong University(上海交通大学);Shanghai Innovation Institute(上海创新研究院);School of Computer Science(计算机科学学院);Shanghai AI Laboratory(上海人工智能实验室)