Foundation models exhibit significant capabilities in decision-making and logical deductions. Nonetheless, a continuing discourse persists regarding their genuine understanding of the world as opposed to mere stochastic mimicry. This paper meticulously examines a simple transformer trained for Othello, extending prior research to enhance comprehension of the emergent world model of Othello-GPT. The investigation reveals that Othello-GPT encapsulates a linear representation of opposing pieces, a factor that causally steers its decision-making process. This paper further elucidates the interplay between the linear world representation and causal decision-making, and their dependence on layer depth and model complexity. We have made the code public.
Evaluating Text Summaries Generated by Large Language Models Using OpenAI's GPT
University of Colorado - Colorado Springs(科罗拉多大学科罗拉多斯普林斯分校);University of Massachusetts Lowell(马萨诸塞大学洛厄尔分校);University of Kansas(堪萨斯大学);DataCrunch Lab, LLC(DataCrunch实验室有限责任公司);University of Florida(佛罗里达大学)
Zero-shot Medical Event Prediction Using a Generative Pre-trained Transformer on Electronic Health Records
Biomedical AI Research Lab, University of California, Los Angeles, Los Angeles, USA(生物医学人工智能研究实验室,加州大学洛杉矶分校);Department of Radiology, University of California, Los Angeles, Los Angeles, USA(放射科,加州大学洛杉矶分校);Department of Bioengineering, University of California, Los Angeles, Los Angeles, USA(生物工程系,加州大学洛杉矶分校);Bioinformatics, University of California, Los Angeles, Los Angeles, USA(生物信息学,加州大学洛杉矶分校);Department of Pediatrics, Division of Immunology, Allergy and Rheumatology, University of California, Los Angeles, Los Angeles, USA(儿科,免疫学、过敏与风湿病学部,加州大学洛杉矶分校);Health Department of Information Services and Solutions, University of California, Los Angeles, University of California, Los Angeles, Los Angeles, USA(信息服务与解决方案健康部门,加州大学洛杉矶分校);Department of Computational Medicine, University of California Los Angeles, Los Angeles, CA, USA(计算医学系,加州大学洛杉矶分校)