Clinical personality assessment screens response validity before interpreting substantive scales. LLM evaluation does not. We apply the validity scaling framework from the PAI and MMPI-3 to metacognitive probe data from 20 frontier models across 524 items. Six validity indices are operationalised: L (maintaining confidence on errors), K (betting on errors), F (withdrawing consensus-endorsed items), Fp (withdrawing correct answers), RBS (inverted monitoring), and TRIN (fixed responding). A tiered classification system identifies four models as construct-level invalid and two as elevated. Valid-profile models produce item-sensitive confidence (mean r = .18, 14 of 16 significant). Invalid-profile models do not (mean r = -.20, d = 2.17, p = .001). Chain-of-thought training produces two opposite response distortions. Two latent dimensions account for 94.6% of index variance. Companion papers extract a portable screening protocol (Cacioli, 2026e) and validate it against selective prediction (Cacioli, 2026f). All data and code: https://github.com/synthiumjp/validity-scaling-llm
Tuning-Free Accountable Intervention for LLM Deployment -- A Metacognitive Approach
Arizona State University(亚利桑那州立大学);University of Science and Technology of China(中国科学技术大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校);MIT(麻省理工学院);Harvard University(哈佛大学)
MeLA: A Metacognitive LLM-Driven Architecture for Automatic Heuristic Design
School of Computer Science, University of Nottingham Ningbo China(诺丁汉大学宁波分校计算机科学学院);College of Teacher Education, Zhejiang Normal University(浙江师范大学教师教育学院)
Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations
Neurosciences Graduate Program University of California San Diego(加州大学圣地亚哥分校神经科学研究生项目);School of Psychology Georgia Tech(佐治亚理工学院心理学系);Department of Psychology New York University(纽约大学心理学系);Department of Neurobiology University of California San Diego(加州大学圣地亚哥分校神经生物学系)
Latent Confidence Alignment for LLM Self-Assessment
Department of Information Management(信息管理系);National Sun Yat-Sen University(国立中山大学);Kaohsiung Medical University Hospital(高雄医学大学附设医院);Kaohsiung Medical University(高雄医学大学)
From Prediction to Justification: Aligning Sentiment Reasoning with Human Rationale via Reinforcement Learning
School of Computer Science and Technology, East China Normal University(东华大学计算机科学与技术学院);Shanghai Qiji Zhifeng Co., Ltd.(上海启智丰科技有限公司);Ocean University of China(中国海洋大学)
Challenging the Validity of Personality Tests for Large Language Models
Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所);Tübingen AI Center(蒂宾根人工智能中心);ETH Zurich(苏黎世联邦理工学院);Methods Center(方法中心);University of Tübingen(蒂宾根大学)
Computational Phenomenology of Borderline Personality Disorder: A Comparative Evaluation of LLM-Simulated Expert Personas and Human Clinical Experts
IDEAS Research Institute(IDEAS研究机构);Adam Mickiewicz University(亚当·密茨凯维奇大学);AMU Center for Artificial Intelligence(AMU人工智能中心);Poznań University of Medical Sciences(波兹南医科大学);Maria Curie-Skłodowska University(玛丽·居里-斯洛多夫斯卡大学);University of Warsaw(华沙大学)