The performance of Large Language Models has achieved superhuman breadth with unprecedented depth. At the same time, the language models are mostly black box models and the underlying mechanisms for performance have been evaluated using synthetic or mechanistic schemes. We extend current mechanistic schemes to incorporate Logic, memory, and nuances of Language such as latent structure. The proposed framework is called LOLAMEME and we provide two instantiations of LOLAMEME: LoLa and MeMe languages. We then consider two generative language model architectures: transformer-based GPT-2 and convolution-based Hyena. We propose the hybrid architecture T HEX and use LOLAMEME framework is used to compare three architectures. T HEX outperforms GPT-2 and Hyena on select tasks.
The Reasoning Trap -- Logical Reasoning as a Mechanistic Pathway to Situational Awareness
MARS 4.0 Fellowship, Cambridge AI Safety Hub(CAISH), University of Cambridge（MARS 4.0 Fellow，剑桥人工智能安全中心（CAISH），剑桥大学）；AWS Generative AI Innovation Center, Amazon Web Services, USA（亚马逊生成AI创新中心，亚马逊网络服务，美国）；Google, USA（谷歌，美国）；Stanford University（斯坦福大学）；Northeastern University, Seattle, WA, USA（东北大学，西雅图，华盛顿州，美国）
From Circuit Evidence to Mechanistic Theory: An Inductive Logic Approach
Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系）；Idiap Research Institute（Idiap研究机构）；CRUK National Biomarker Centre, University of Manchester（曼彻斯特大学癌症研究联盟国家生物标志物中心）
Mechanistic Evidence for Faithfulness Decay in Chain-of-Thought Reasoning
Fordham University（福特汉姆大学）；Algoverse AI Research（Algoverse AI研究）；Rice University（稻子大学）；UC San Diego（圣地亚哥大学）；Santa Clara University（圣克拉拉大学）；LMU Munich（慕尼黑路德维希-马克西米利安大学）
Compliance vs. Sensibility: On the Reasoning Controllability in Large Language Models
School of Computer Science, University of Sheffield（谢菲尔德大学计算机科学学院）；School of EECS, Queen Mary University of London（伦敦女王学院电子工程与计算机科学学院）；The Alan Turing Institute（艾伦·图灵研究所）
Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inference
Department of Computer Science, University of Manchester, UK（英国曼彻斯特大学计算机科学系）；Idiap Research Institute, Switzerland（瑞士Idiap研究所）；School of Computer Science, University of Sheffield, UK（英国谢菲尔德大学计算机科学学院）；National Biomarker Centre, CRUK-MI, University of Manchester, UK（英国曼彻斯特大学国家生物标志物中心）
Adaptive Constraint Propagation: Scaling Structured Inference for Large Language Models via Meta-Reinforcement Learning
Department of Computer Science, Iowa State University（计算机科学系，爱荷华州立大学）；Department of Civil, Construction & Environmental Engineering, Iowa State University（土木、建设与环境工程系，爱荷华州立大学）
Perturbation-based Regional Interpretability through Subtraction Mapping (PRISM): naming-error dissociations in language models and post-stroke aphasia
University of South Carolina（南卡罗来纳大学）；ALLT.AI, LLC（ALLT.AI有限责任公司）；USC School of Medicine（南卡罗来纳大学医学院）
Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gates
Hang Chen School of Computer Science and Technology Xi’an Jiaotong University（Hang Chen 计算机科学与技术学院 西安交通大学）；Jiaying Zhu School of Computer Science and Engineering The Chinese University of Hong Kong（Jiaying Zhu 计算科学与工程学院 香港中文大学）；Xinyu Yang School of Computer Science and Technology Xi’an Jiaotong University（Xinyu Yang 计算机科学与技术学院 西安交通大学）；Wenya Wang School of Computer Science and Engineering Nanyang Technological University（Wenya Wang 计算科学与工程学院 新加坡国立大学）
