In autoregressive large language models (LLMs), temporal straightening offers an account of how the next-token prediction objective shapes representations. Models learn to progressively straighten the representational trajectory of input sequences across layers, potentially facilitating next-token prediction via linear extrapolation. However, a direct link between this trajectory and token-level behavior has been missing. We provide such a link by relating contextual curvature-a geometric measure of how sharply the representational trajectory bends over recent context-to next-token entropy. Across two models (GPT-2 XL and Pythia-2.8B), contextual curvature is correlated with entropy, and this relationship emerges during training. Perturbation experiments reveal selective dependence: manipulating curvature through trajectory-aligned interventions reliably modulates entropy, while geometrically misaligned perturbations have no effect. Finally, regularizing representations to be straighter during training modestly reduces token-level entropy without degrading validation loss. These results identify trajectory curvature as a task-aligned representational feature that influences behavioral uncertainty in LLMs.
From Associations to Activations: Comparing Behavioral and Hidden-State Semantic Geometry in LLMs
Computational Modelling Lab, Humboldt-Universität zu Berlin, Berlin, Germany(洪堡-柏林大学计算建模实验室);Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany(柏林自由大学教育与心理学系);Department for AI in Society, Science, and Technology, Zuse Institute Berlin, Germany(柏林祖克研究所人工智能与社会、科学与技术部门);Institute of Mathematics, Technische Universitat Berlin, Germany(柏林技术大学数学研究所)
Auditing Framing-Sensitive Behavioral Instability in Large Language Models for Mental Health Interactions
a School of computer science, digital engineering and AI, Long Island University, Brooklyn, NY, USA(a 计算科学与数字工程及人工智能学院,罗格斯大学,布鲁克林,纽约,美国);b Department of Psychology, Long Island University, Brooklyn, NY, USA(b 心理学系,罗格斯大学,布鲁克林,纽约,美国)
Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter It
Boston University(波士顿大学);University of Amsterdam(阿姆斯特丹大学);Umeå University(乌梅大学);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);TTIC(TTIC研究中心);The University of Texas at Austin(德克萨斯大学奥斯汀分校)
Large language models reorganize representational geometry during in-context learning
School of Psychological and Brain Sciences, Georgia Tech(佐治亚理工学院心理与脑科学学院);Department of Psychology, New York University(纽约大学心理学系);Center of Excellence for Computational Cognition, Georgia Tech(佐治亚理工学院计算认知卓越中心);Honda Research Institute(本田研究院);Departments of Neuroscience and Psychology, The University of Texas at Austin(德克萨斯大学奥斯汀分校神经科学与心理学系)
Behind Harmful Compliance: Behavioral and Mechanistic Divergence Across LLM Jailbreaks
Department of Computer Science(计算机科学系);Luddy School of Informatics, Computing, and Engineering(信息学、计算与工程学院);Indiana University Bloomington(印第安纳大学布卢明顿分校)
Distinguishing representational geometries with controversial stimuli: Bayesian experimental design and its application to face dissimilarity judgments
Zuckerman Mind Brain Behavior Institute(祖克曼心智脑行为研究所);Columbia University(哥伦比亚大学)
Epistemic Familiarity is Associated With Belief Stability in Large Language Models
Khoury College of Computer Sciences, Northeastern University(东北大学计算机科学学院);Network Science Institute, Northeastern University(东北大学网络科学研究所);Santa Fe Institute(圣达菲研究所)
Defense Against LLM Backdoors using Critical Neuron Isolation Pruning
Huazhong University of Science and Technology(华中科技大学);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Nanyang Technological University(南洋理工大学)
Theory of Mind and Self-Attributions of Mentality are Dissociable in LLMs
Google, Paradigms of Intelligence Team(Google,智能范式团队);Knowledge Lab, University of Chicago(芝加哥大学知识实验室);Institute of Philosophy, School of Advanced Study, University of London(伦敦大学高等研究院哲学研究所);Department of Biomedical Informatics and Medical Education and Department of Bioethics and Humanities, School of Medicine, University of Washington(华盛顿大学医学院生物医学信息学与医学教育系及生物伦理学与人文学系);Kellogg School of Management, Northwestern University(西北大学凯洛格管理学院);Santa Fe Institute(圣塔菲研究所)
Department of Statistics and Operations Research, UNC-Chapel Hill(统计与运筹学系,北卡罗来纳大学 Chapel Hill 分校);Imperial College Business School, Imperial College London(帝国理工学院伦敦校区商学院)
Hierarchical Frequency Tagging Probe (HFTP): A Unified Approach to Investigate Syntactic Structure Representations in Large Language Models and the Human Brain
Peking University(北京大学);Zhejiang University(浙江大学);Beijing Language and Culture University(北京语言大学);Beijing Institute for General Artificial Intelligence(北京通用人工智能研究院)