While large language models (LLMs) have seen unprecedented advancements in capabilities and applications across a variety of use-cases, safety alignment of these models is still an area of active research. The fragile nature of LLMs, even models that have undergone extensive alignment and safety training regimes, warrants additional safety steering steps via training-free, inference-time methods. While recent work in the area of mechanistic interpretability has investigated how activations in latent representation spaces may encode concepts, and thereafter performed representation engineering to induce such concepts in LLM outputs, the applicability of such for safety is relatively under-explored. Unlike recent inference-time safety steering works, in this paper we explore safety steering of LLM outputs using: (i) category-specific steering vectors, thereby enabling fine-grained control over the steering, and (ii) sophisticated methods for extracting informative steering vectors for more effective safety steering while retaining quality of the generated text. We demonstrate our exploration on multiple LLMs and datasets, and showcase the effectiveness of the proposed steering method, along with a discussion on the implications and best practices.
PIXEL: Adaptive Steering Via Position-wise Injection with eXact Estimated Levels under Subspace Calibration
University of Queensland(昆士兰大学);Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)(穆罕默德·本·扎耶德人工智能大学);Provable Responsible AI and Data Analytics (PRADA) Lab(可证责任AI与数据分析师实验室);King Abdullah University of Science and Technology (KAUST)(国王 Abdullah 科学与技术大学)
Steering at the Source: Style Modulation Heads for Robust Persona Control
The University of Tokyo, Tokyo, Japan(东京大学);National Institute of Informatics Research(信息处理研究所);Development Center for Large Language Models, Japan(大型语言模型发展中心)
FaithSteer-BENCH: A Deployment-Aligned Stress-Testing Benchmark for Inference-Time Steering
University of Electronic Science and Technology of China(电子科技大学);Mohamed bin Zayed University of Artificial Intelligence(马尔代夫人工智能大学);South China University of Technology(华南理工大学)
CogSteer: Cognition-Inspired Selective Layer Intervention for Efficiently Steering Large Language Models
Department of Informatics, Universität Hamburg(汉堡大学信息学院);The University of Sydney(悉尼大学);Alibaba International Digital Commerce(阿里巴巴国际数字商业);Institute of Psychology, Chinese Academy of Sciences(中国科学院心理研究所)
Beyond Visual Enhancement: Adaptive Multi-Context Steering to Mitigate LVLM Hallucinations
Shanghai Jiao Tong University(上海交通大学);Peking University(北京大学);Beijing University of Chemical Technology(北京化工大学);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);University of Science and Technology of China(中国科学技术大学);Tsinghua University(清华大学)
Northeastern University(东北大学);Washington University in St. Louis(圣路易斯华盛顿大学);Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Tsinghua University(清华大学);Security Capability Center(360安全能力中心);Tencent(腾讯);Beijing University of Posts and Telecommunications(北京邮电大学)
Listwise Preference Optimization with Element-wise Confusions for Aspect Sentiment Quad Prediction
The Hong Kong Polytechnic University(香港理工大学);Lingnan University(岭大);University of Technology Sydney(悉尼大学);The Education University of Hong Kong(香港教育大学)
Is Inference Mediated by Distinct Semantic Structures in LLMs? A Mechanistic Interpretation
University of Manchester(曼彻斯特大学);University of Sheffield(谢菲尔德大学);Idiap Research Institute(Idiap研究所);CRUK National Biomarker Centre, University of Manchester(CRUK国家生物标志物中心,曼彻斯特大学)
Internal Chain-of-Thought: Empirical Evidence for Layer-wise Subtask Scheduling in LLMs
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Southeast University(东南大学)
CLaS-Bench: A Cross-Lingual Alignment and Steering Benchmark
Saarland University(萨尔兰大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI));Centre for European Research in Trusted AI (CERTAIN)(可信AI欧洲研究中心(CERTAIN));University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);TU Darmstadt(德累斯顿技术大学);Mila - Quebec Artificial Intelligence Institute(魁北克人工智能研究所(Mila));McGill University(麦吉尔大学)
Dynamically Updating Event Representations for Temporal Relation Classification with Multi-category Learning
Graduate School of Informatics, Kyoto University(京都大学信息学研究科);National Institute for Japanese Language and Linguistics(国立国语研究所);Ochanomizu University(御茶水女子大学)