一篇早期论文(Hong, Potteiger, and Zapata 2026)表明,未经优化的GPT 4.1提示在67%的情况下能从开放式调查文本中预测粉丝报告的经验评分。本文测试了提示设计和模型选择对性能的相对影响。我们比较了四个配置,涉及约10,000份赛后调查问卷:原始基线提示和一个中等定制版本,结合三个GPT模型(4.1、4.1-mini、5.2)。提示定制使GPT 4.1的±1内一致率提高了约2个百分点(从67%到69%)。从最佳配置切换模型会降低性能:GPT 5.2回到基线,GPT 4.1-mini下降了6个百分点。两者结合的效果远不及输入本身:在有能力的配置中,准确性在文本语言特征上的变化比提示或模型的选择大一个数量级。上限有两个部分。一个是模型读取文本的偏差,提示设计可以纠正。另一个是粉丝写的和他们实际决定的差异,工程无法关闭,因为缺失信息不在文本中。提示定制影响了第一部分,模型选择无法可靠地影响。结果不是提示工程帮助一点,而是提示工程在特定且可预测的方式上帮助,仅在它能到达的上限部分。
英文摘要:
An earlier paper (Hong, Potteiger, and Zapata 2026) established that an unoptimized GPT 4.1 prompt predicts fan-reported experience ratings within one point 67% of the time from open-ended survey text. This paper tests the relative impact of prompt design and model selection on that performance. We compared four configurations on approximately 10,000 post-game surveys from five MLB teams: the original baseline prompt and a moderately customized version, crossed with three GPT models (4.1, 4.1-mini, 5.2). Prompt customization added roughly two percentage points of within +/-1 agreement on GPT 4.1 (from 67% to 69%). Both model swaps from that best configuration degraded performance: GPT 5.2 returned to the baseline, and GPT 4.1-mini fell six percentage points below it. Both levers combined were dwarfed by the input itself: across capable configurations, accuracy varied more than an order of magnitude more by the linguistic character of the text than by the choice of prompt or model. The ceiling has two parts. One is a bias in how the model reads text, which prompt design can correct. The other is a difference between what fans write about and what they actually decide, which no engineering can close because the missing information is not in the text. Prompt customization moved the first part; model selection moved neither reliably. The result is not that "prompt engineering helps a little" but that prompt engineering helps in a specific and predictable way, on the part of the ceiling it can reach.
When Rating Scales Fall Short: LLM-Assisted Discovery of ADHD Signals in Turkish Teacher Narratives
Department of Computer Science, University of Illinois Chicago(伊利诺伊大学芝加哥分校计算机科学系);Department of Child and Adolescent Psychiatry, Gazi University(加齐大学儿童与青少年精神病学系)
G-Zero: Self-Play for Open-Ended Generation from Zero Data
Washington University in St. Louis(华盛顿大学圣路易斯分校);University of Virginia(弗吉尼亚大学);University of Maryland(马里兰大学);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)
Comparing Large Language Model AI and Human-Generated Coaching Messages for Behavioral Weight Loss
Northeastern University(东北大学);VA Connecticut Healthcare System(康涅狄格州医疗系统);Portland Psychotherapy Clinic, Research and Training Center(波特兰心理治疗诊所、研究与培训中心);University of Washington(华盛顿大学);The University of Melbourne(墨尔本大学);Drexel University(德雷塞尔大学)
Extracting Consumer Insight from Text: A Large Language Model Approach to Emotion and Evaluation Measurement
Monash University(莫纳什大学);University of New South Wales(新南威尔士大学);Flinders University(弗林德斯大学);Babson College(巴布森学院);University of Bath(巴斯大学);Tecnológico de Monterrey(蒙特雷理工学院)
Interpretable, Fairly Evaluated Automated L2 Speaking Assessment that Beats the Single-Human Ceiling and Why Pause Encoding Does Not Change LLM Fluency Scores
Before and After Temperature: A Distributional View of Creative LLM Generation
University of California, San Diego(加州大学圣地亚哥分校);Indian Institute of Technology, Kharagpur(印度理工学院克哈格布尔分校);Delhi Technological University(德里技术大学);Carnegie Mellon University(卡内基梅隆大学);Rutgers University(罗格斯大学);Georgia Institute of Technology(佐治亚理工学院)
Judge Circuits Explain Format-Induced Inconsistency in LLM-as-a-Judge
Technische Universität Berlin(柏林技术大学);BIFOLD – Berlin Institute for the Foundations of Learning and Data(柏林学习与数据基础研究院);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Fraunhofer Heinrich Hertz Institute(弗劳恩霍夫海因里希·赫茨研究所);Marburg University(马尔堡大学);Centre for European Research in Trusted AI (CERTAIN)(欧洲可信人工智能研究中心)
Secure On-Premise Deployment of Open-Weights Large Language Models in Radiology: An Isolation-First Architecture with Prospective Pilot Evaluation
Department of Diagnostic and Interventional Radiology, University Hospital Bonn(诊断与介入放射科,波恩大学医院);Department of Research-IT, University Hospital Bonn(研究-IT部门,波恩大学医院)
Tailoring AI-Driven Reading Scaffolds to the Distinct Needs of Neurodiverse Learners
Sony Computer Sciences Laboratories, Paris, France(索尼计算机科学实验室(巴黎));Institut de l’Audition - Institut Pasteur, Paris, France(听觉研究所 - 巴斯德研究所(巴黎));Université Paris Cité, France(巴黎西岱大学);Centro Ricerca e Cura di Roma, Italy(罗马研究与治疗中心);Sapienza Università di Roma, Italy(罗马大学);I-Lab, Rome, Italy(I-Lab(罗马))