将自我一致性融入言语自信:Gemma 3 4B上的预注册负结果及事后补救
Distilling Self-Consistency into Verbal Confidence: A Pre-Registered Negative Result and Post-Hoc Rescue on Gemma 3 4B
AI总结:
研究发现,小规模指令微调LLM在最小激发下产生退化言语自信,通过自我一致性衍生目标的自信条件监督微调未能缩小内部信息与言语输出间的差距,但事后去除过滤器后在TriviaQA上提升了AUROC2至0.774,支持目标依赖性解释。
摘要:
小规模指令微调的LLM在最小激发下产生退化言语自信:天花板率超过95%,接近偶然的类型2 AUROC,以及无效的有效性曲线。我们测试自信条件监督微调(CSFT)结合自我一致性衍生目标能否缩小内部信息与言语输出间的差距。在Gemma 3 4B-it上预注册的Phase 0协议,通过限制训练到具有正确模态答案的项目产生负结果:AUROC2从0.554降至0.509,由于训练目标的标签熵崩溃。探索性补救去除了过滤器,训练所有2000个校准项目。这产生了二元言语正确性判别器,在held-out TriviaQA上AUROC2=0.774,将10样本自我一致性信号(AUROC2=0.999)压缩为单次通过输出,超过logit熵(0.701)。洗牌目标对照无改善(0.501)。在MMLU上,准确性从54.2%提升至77.4%,与基线洗牌模型(56.1%)相比,支持目标依赖性解释。结果是探索性的,二元而非连续校准的,且在单一尺度上观察到。它识别出两个设计教训:自信训练需要标签熵,正确目标正则化输出格式。
英文摘要:
Small instruct-tuned LLMs produce degenerate verbal confidence under minimal elicitation: ceiling rates above 95%, near-chance Type-2 AUROC, and Invalid validity profiles. We test whether confidence-conditioned supervised fine-tuning (CSFT) with self-consistency-derived targets can close the gap between internal information and verbal readout. A pre-registered Phase 0 protocol on Gemma 3 4B-it with a modal filter restricting training to items with correct modal answers produced a negative result: AUROC2 dropped from 0.554 to 0.509 due to label-entropy collapse in the training targets. An exploratory rescue removed the filter, training on all 2,000 calibration items. This produced a binary verbal correctness discriminator with AUROC2 = 0.774 on held-out TriviaQA, compressing a 10-sample self-consistency signal (AUROC2 = 0.999) into a single-pass readout exceeding logit entropy (0.701). The shuffled-target control showed no improvement (0.501). On MMLU, accuracy improved from 54.2% to 77.4% with the shuffled model at baseline (56.1%), supporting a target-dependent interpretation. The result is exploratory, binary rather than continuously calibrated, and observed at a single scale. It identifies two design lessons: confidence training requires label entropy, and correct targets regularise output format.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。