AntiPaSTO:通过反平行表示实现自监督诚实引导
AntiPaSTO: Self-Supervised Honesty Steering via Anti-Parallel Representations
AI总结:
AntiPaSTO通过反平行表示实现自监督诚实引导,通过对比学习提升模型在日常困境中的引导效果,并在多个价值轴上取得优势。
摘要:
随着模型能力的提升,人类无法可靠地验证其输出。可扩展的引导需要内部、自监督且能迁移至分布外的 方法;现有方法满足其中一些但不全部。我们引入AntiPaSTO,通过反平行轴分离表示(+1/-1产生相反位移),通过一致性约束防止崩溃。训练仅使用两个对比词插入模板句子,无需偏好标签。当我们在Gemma-3-1B上使用800个此类合成对时,AntiPaSTO在DailyDilemmas任务中比提示基线提高了6.9倍的Steering F1,并在6个测试的价值轴中胜出5个。我们还发现初步证据表明,它在提示触发拒绝时能保持双向控制。
英文摘要:
As models grow more capable, humans cannot reliably verify what they say. Scalable steering requires methods that are internal, self-supervised, and transfer out-of-distribution; existing methods satisfy some but not all three. We introduce AntiPaSTO, which separates representations along an antiparallel axis (+1/-1 produce opposite shifts), with coherence constraints preventing collapse. Training uses only two contrasting words inserted into template sentences, with no preference labels. When we use 800 such synthetic pairs on Gemma-3-1B, AntiPaSTO beats prompting baselines by 6.9x Steering F1 on DailyDilemmas and wins on 5 of 6 tested value axes. We also find preliminary evidence that it maintains bidirectional control where prompting triggers refusal.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。