从前提到预测:通过受控诱导验证一种几何幻觉分类法
From Prerequisites to Predictions: Validating a Geometric Hallucination Taxonomy Through Controlled Induction
AI总结:
本文通过受控诱导验证了一种几何幻觉分类法,发现覆盖缺口幻觉在几何上最为显著,而类型1和2的非分离在大规模模型中真实存在。
摘要:
我们测试了一种几何幻觉分类法——将失败分为中心漂移(类型1)、错误井收敛(类型2)或覆盖缺口(类型3)——是否能通过在GPT-2中的受控诱导区分幻觉类型。使用双层统计设计,以提示(每组N=15)为推理单位,我们用不同的生成种子运行每个实验20次以量化结果稳定性。在静态嵌入中,类型3的范数分离是稳健的(在18/20次运行中显著,Holm校正后在14/20次运行中显著,中位数r=+0.61)。在上下文隐藏状态中,类型3的范数效应方向是稳定的(19/20次运行),但N=15时功率不足(在4/20次运行中显著,中位数r=-0.28)。类型1和类型2在两种空间中均不分离(≤3/20次运行)。在词级别测试中,伪重复使显著性提高了4-16倍——这一发现被所有20次运行所复制。结果证实覆盖缺口幻觉是最重要的几何上区分的失败模式,其特征在于幅度而非方向,并确认在124M参数下类型1/2的非分离是真实的。
英文摘要:
We test whether a geometric hallucination taxonomy -- classifying failures as center-drift (Type~1), wrong-well convergence (Type~2), or coverage gaps (Type~3) -- can distinguish hallucination types through controlled induction in GPT-2. Using a two-level statistical design with prompts ($N = 15$/group) as the unit of inference, we run each experiment 20 times with different generation seeds to quantify result stability. In static embeddings, Type~3 norm separation is robust (significant in 18/20 runs, Holm-corrected in 14/20, median $r = +0.61$). In contextual hidden states, the Type~3 norm effect direction is stable (19/20 runs) but underpowered at $N = 15$ (significant in 4/20, median $r = -0.28$). Types~1 and~2 do not separate in either space (${\leq}\,3/20$ runs). Token-level tests inflate significance by 4--16$\times$ through pseudoreplication -- a finding replicated across all 20 runs. The results establish coverage-gap hallucinations as the most geometrically distinctive failure mode, carried by magnitude rather than direction, and confirm the Type~1/2 non-separation as genuine at 124M parameters.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。