arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.39766cs.LG
收藏

激活探针在少量正例下仍可靠

Activation Probes Are Reliable With a Handful of Positive Examples

Massachusetts Institute of Technology(麻省理工学院)

Riya Tyagi, Stefan Heimersheim

AI总结:

针对AI对齐错误监测的少量正例数据挑战,研究发现激活探针在少量正例下可靠,大量负例训练、LLM适度上采样及更大模型可提升正例效率,为监测罕见对齐错误提供方法。

摘要:

监测高级AI以发现罕见的对齐错误面临数据挑战:存在大量对齐示例,但仅有少量未对齐示例。我们在“少量正例vs数千负例”的场景下,针对垃圾邮件检测和诚实性检测任务测试激活探针。对于我们的任务,当正例数量较少(1-10个)时,使用大量负例训练平均而言比平衡训练更具正例效率。我们还发现,LLM上采样可提供相当于约两倍真实正例数量的性能提升,不过过度上采样会损害性能。最后,我们观察到正例缩放趋势,即更大的模型在探针任务中对正例的效率更高。我们的发现表明,应利用可用的大量负例来放大罕见但关键的对齐错误示例的信号。

英文摘要:

Efforts to monitor advanced AI for rare misalignments face a data challenge: abundant aligned examples but only a handful of misaligned ones. We test activation probes in this "few vs. thousands" regime on spam and honesty detection tasks. For our tasks, training with many negative examples is on average more positive-sample-efficient than balanced training for small numbers (1-10) of positive samples. We also find that LLM upsampling can provide a performance boost equivalent to roughly doubling the number of real positive samples, though excessive upsampling hurts performance. Finally, we show a positive scaling trend, where larger models are more positive-sample-efficient to probe. Our findings suggest we should leverage the large number of negative samples available to amplify the signal from rare but critical misalignment examples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑