Large language models (LLMs) can generate long-form and coherent text, yet they often hallucinate facts, which undermines their reliability. To mitigate this issue, inference-time methods steer LLM representations toward the "truthful directions" previously learned for truth elicitation. However, applying these truthful directions with the same intensity fails to generalize across different query contexts. We propose LITO, a Learnable Intervention method for Truthfulness Optimization that automatically identifies the optimal intervention intensity tailored to each specific context. LITO explores a sequence of model generations based on increasing levels of intervention intensities. It selects the most accurate response or refuses to answer when the predictions are highly uncertain. Experiments on multiple LLMs and question-answering datasets demonstrate that LITO improves truthfulness while preserving task accuracy. The adaptive nature of LITO counters the limitations of one-size-fits-all intervention methods, maximizing truthfulness by reflecting the model's internal knowledge only when it is confident. Our code is available at https://github.com/launchnlp/LITO.
TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of Chinese Academy of Sciences(中国科学院大学);Key Laboratory of AI Safety, Chinese Academy of Sciences(中国科学院人工智能安全重点实验室)
How Post-Training Reshapes LLMs: A Mechanistic View on Knowledge, Truthfulness, Refusal, and Confidence
University of California, Los Angeles(加州大学洛杉矶分校);University of Alberta(阿尔伯塔大学);University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Harvard University(哈佛大学)
TruthPrInt: Mitigating Large Vision-Language Models Object Hallucination Via Latent Truthful-Guided Pre-Intervention
Drexel University(德雷塞尔大学);University of Electronic Science and Technology of China(电子科技大学);Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));LLNL(劳伦斯利弗莫尔国家实验室);Lehigh University(莱斯大学)