让语言模型产生感知的词汇
Words That Make Language Models Perceive
AI总结：
本研究验证显式感官提示可挖掘纯文本大语言模型中由语言多模态规律塑造的潜在感知结构，通过轻量级提示工程就能激活对应模态的表征，使其与专业视觉、音频编码器的表征更一致。
摘要：
纯文本训练的大语言模型（LLMs）表面上缺乏任何直接感知经验，但其内部表征会受到语言中编码的多模态规律的隐性塑造。我们验证了如下假设：显式感官提示能够挖掘这种潜在结构，使纯文本LLM与专业视觉、音频编码器的表征更趋一致。当感官提示指令模型“看”或“听”时，会引导模型基于从未实际提供的潜在视觉或听觉证据来解析下一词元预测。研究结果表明，轻量级提示工程能够可靠地激活纯文本训练LLM中对应模态的表征。
英文摘要：
Large language models (LLMs) trained purely on text ostensibly lack any direct perceptual experience, yet their internal representations are implicitly shaped by multimodal regularities encoded in language. We test the hypothesis that explicit sensory prompting can surface this latent structure, bringing a text-only LLM into closer representational alignment with specialist vision and audio encoders. When a sensory prompt tells the model to 'see' or 'hear', it cues the model to resolve its next-token predictions as if they were conditioned on latent visual or auditory evidence that is never actually supplied. Our findings reveal that lightweight prompt engineering can reliably activate modality-appropriate representations in purely text-trained LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
