arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.03708cs.CLcs.AIstat.ML
收藏

迈向对AI模型的通用引导与监控

Toward universal steering and monitoring of AI models

Computer Science and Engineering(计算机科学与工程);Broad Institute of MIT and Harvard(MIT和哈佛大学Broad研究所);UC San Diego(圣地亚哥大学);Harvard SEAS(哈佛大学工程与应用科学学院);MIT Mathematics(MIT数学系);Halıcıoğlu Data Science Institute(Halıcıoğlu数据科学研究所);Harvard CMSA(哈佛大学计算机科学与应用数学系)

Daniel Beaglehole, Adityanarayanan Radhakrishnan, Enric Boix-Adserà, Mikhail Belkin

AI总结:

本文提出一种可扩展方法提取大规模AI模型的通用概念线性表示,证明其能实现跨语言和多概念引导以提升能力并缓解不对齐行为,且基于该表示的预测模型在监控幻觉等不对齐内容上比直接判断输出更准确。

摘要:

现代AI模型包含了大量人类知识,但对其内部知识表示的理解仍然难以捉摸。表征这种表示的结构和属性将有助于提升模型能力并开发有效的安全防护措施。基于特征学习的最新进展,我们开发了一种有效且可扩展的方法,用于在大规模AI模型(语言模型、视觉-语言模型和推理模型)中提取通用概念的线性表示。我们展示了这些表示如何实现模型引导,通过引导我们暴露了漏洞,缓解了不对齐行为,并提升了模型能力。此外,我们证明了概念表示在人类语言之间具有显著的跨语言可迁移性,并且可以组合以实现多概念引导。通过对数百个概念的定量分析,我们发现较新、较大的模型更具可引导性,且引导能够超越标准提示进一步提升模型能力。我们展示了概念表示如何有效监控不对齐内容(幻觉、有毒内容)。我们证明,使用概念表示构建的预测模型在监控不对齐内容方面,比直接判断输出的模型更准确。总而言之,我们的结果阐明了利用内部表示来映射AI模型中知识、推进AI安全并提升模型能力的强大作用。

英文摘要:

Modern AI models contain much of human knowledge, yet understanding of their internal representation of this knowledge remains elusive. Characterizing the structure and properties of this representation will lead to improvements in model capabilities and development of effective safeguards. Building on recent advances in feature learning, we develop an effective, scalable approach for extracting linear representations of general concepts in large-scale AI models (language models, vision-language models, and reasoning models). We show how these representations enable model steering, through which we expose vulnerabilities, mitigate misaligned behaviors, and improve model capabilities. Additionally, we demonstrate that concept representations are remarkably transferable across human languages and combinable to enable multi-concept steering. Through quantitative analysis across hundreds of concepts, we find that newer, larger models are more steerable and steering can improve model capabilities beyond standard prompting. We show how concept representations are effective for monitoring misaligned content (hallucinations, toxic content). We demonstrate that predictive models built using concept representations are more accurate for monitoring misaligned content than using models that judge outputs directly. Together, our results illustrate the power of using internal representations to map the knowledge in AI models, advance AI safety, and improve model capabilities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑