Efforts to monitor advanced AI for rare misalignments face a data challenge: abundant aligned examples but only a handful of misaligned ones. We test activation probes in this "few vs. thousands" regime on spam and honesty detection tasks. For our tasks, training with many negative examples is on average more positive-sample-efficient than balanced training for small numbers (1-10) of positive samples. We also find that LLM upsampling can provide a performance boost equivalent to roughly doubling the number of real positive samples, though excessive upsampling hurts performance. Finally, we show a positive scaling trend, where larger models are more positive-sample-efficient to probe. Our findings suggest we should leverage the large number of negative samples available to amplify the signal from rare but critical misalignment examples.
Detection Without Correction: A Robust Asymmetry in Activation-Based Hallucination Probing
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India(印度理工学院帕纳分校计算机科学与工程系);Department of Computer Science, Rajarshi School of Management and Technology, Varanasi, India(印度瓦拉纳西 Rajarshi 管理与技术学校计算机科学系);Department of Electronics and Communication Engineering, Jaypee Institute of Information Technology, Noida, India(印度诺伊达 Jaypee 信息科技学院电子与通信工程系)
Linear Separability of Activation Representations after Supervised Fine-Tuning on Incorrect Responses: A Study of Synthetic Dishonesty in Large Language Models
Algoverse AI Research;Medical Sciences Education Research Center, Mashhad University of Medical Sciences(马什哈德大学医学科学教育研究中心);Student Research Committee, Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences(谢赫·贝赫什提大学医学科学学院学生研究委员会,健康信息科技与管理系,医学信息学)