Don't Ask: Understanding What Numeric Self-Reports Measure in Language Models
Universidad de Buenos Aires. Facultad de Ciencias Exactas y Naturales. Departamento de Computación(布宜诺斯艾利斯大学。精确与自然科学学院。计算机系);CONICET-Universidad de Buenos Aires. Instituto de Ciencias de la Computación (ICC)(国家科学与技术研究理事会-布宜诺斯艾利斯大学。计算机科学研究所)
Nicolas Martorell, Bruno Bianchi
备注:
41 pages, 30 figures, 18 tables. Replaces the preprint "Quantitative Introspection in Language Models: Tracking Emotive States Across Conversation" with an extended, revised study: new title; 8 models (previously 1 in depth, 4 in less detail); 100 conversations (40 previous, 60 new); probe readouts before and after the rating question (previously only after); new controls; revised conclusions
Language models are often asked to rate their internal states with a number, but it is unclear how reliable this measure is. We train probes for four affective concepts (wellbeing, interest, focus and impulsivity) in eight instruction-tuned models to track these states over multi-turn conversations. We compare each probe's score with a numeric self-report of the same state, elicited at every turn. We find that rating questions displace the probed state, and that the rating is only weakly coupled with the probe score before the question but tracks the post-question probe much more strongly. Steering experiments suggest that models use the affective directions to inform ratings, but that relationship is not specific: ratings are associated with other concepts' probes as much as with their own, and change little when the question is about the user. Finally, neither the pre-question coupling nor concept specificity improves with scale (1B-27B). We warn about the effect a question can have on the internal states one is asking about, and caution against numeric self-report as a reliable measure of those states.
Only Ask What You Don't Know: Grounded Delta Planning for Efficient Multi-step RAG
Dept. of Computer Science and Information Engineering, National Taiwan University(国立台湾大学资讯工程学系);Graduate Institute of Communication Engineering, National Taiwan University(国立台湾大学电信工程学研究所);Department of Economics, National Taiwan University(国立台湾大学经济学系);Department of Information Management, National Taiwan University(国立台湾大学资讯管理学系);NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE)(国立台湾大学人工智能卓越研究中心)
AskSport: Web Application for Sports Question-Answering
Faculty of Computing(计算机学院);Institute of Mathematics and Computer Sciences(数学与计算机科学研究所);Federal University of Uberlandia(乌贝兰迪亚联邦大学);University of Sao Paulo(圣保罗大学)
Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamics
Carnegie Mellon University(卡内基梅隆大学);Boston University(波士顿大学);Harvard University(哈佛大学);University of Oxford, Martian(牛津大学,火星);Max Planck Institute for Software Systems(马克斯·普朗克软件系统研究所)