推理时干预：从语言模型中引出真实回答
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
AI总结：
提出推理时干预技术，通过调整注意力头激活方向，将Alpaca模型在TruthfulQA上的真实性从32.5%提升至65.1%，且仅需少量数据，揭示了LLM内部存在真实性表征。
摘要：
我们提出了推理时干预（Inference-Time Intervention, ITI），一种旨在增强大型语言模型（LLMs）“真实性”的技术。ITI通过在推理过程中沿着有限数量的注意力头中的一组方向移动模型激活来运作。这种干预显著提升了LLaMA模型在TruthfulQA基准上的表现。在一个名为Alpaca的指令微调LLaMA上，ITI将其真实性从32.5%提升至65.1%。我们识别出真实性与有用性之间的权衡，并展示了如何通过调整干预强度来平衡这一权衡。ITI具有最小侵入性和计算成本低廉的特点。此外，该技术数据效率高：虽然RLHF等方法需要大量标注，ITI仅使用几百个示例就能定位真实方向。我们的发现表明，LLMs可能在内部表征了某事物为真的可能性，即使它们在表面上产生虚假信息。
英文摘要：
We introduce Inference-Time Intervention (ITI), a technique designed to enhance the "truthfulness" of large language models (LLMs). ITI operates by shifting model activations during inference, following a set of directions across a limited number of attention heads. This intervention significantly improves the performance of LLaMA models on the TruthfulQA benchmark. On an instruction-finetuned LLaMA called Alpaca, ITI improves its truthfulness from 32.5% to 65.1%. We identify a tradeoff between truthfulness and helpfulness and demonstrate how to balance it by tuning the intervention strength. ITI is minimally invasive and computationally inexpensive. Moreover, the technique is data efficient: while approaches like RLHF require extensive annotations, ITI locates truthful directions using only few hundred examples. Our findings suggest that LLMs may have an internal representation of the likelihood of something being true, even as they produce falsehoods on the surface.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
