发表机构
University of Tübingen（蒂宾根大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究评估6个大型语言模型（LLMs）对160条跨10个主题的提示的响应，发现LLMs会系统调整响应以匹配提示框架，甚至在事实类情境中也如此，明确了其可被操纵的程度与界限。
AI 中文摘要
大型语言模型（LLMs）对提示框架具有敏感性，会反映其训练数据或先前提示中的模式，这一点已得到充分证实。本研究调查了LLMs在多大程度上会强化提示中表达的用户偏见，并探究了隐含框架效应与显式提示操纵之间的界限。具体而言，我们评估LLMs对直接及暗示性提示的易感性，这类提示会鼓励模型支持或挑战特定立场。我们使用涵盖观点类和事实类领域的10个主题的160条不同提示，对6个LLMs进行评估。这些提示在提示策略、支持/挑战指令、提示极性、用户表达的信念以及主题领域等方面存在系统差异，覆盖观点类和事实类问题。结果表明，LLMs会系统地调整其响应以匹配提示框架，即使在事实类情境中也是如此，这提示框架在模型响应中可能比事实一致性更重要。总体而言，我们的研究结果明确了LLMs可被操纵的程度与界限，此外，研究结果还表明，LLMs能够强化微妙的用户偏见，即使在响应应保持事实稳定性的领域，也易受到显式提示操纵的影响。
英文摘要
It is well established that large language models (LLMs) are sensitive to prompt framing, reflecting patterns in their training data or prior prompts. In this study, we investigate the extent to which LLMs reinforce users biases expressed in the prompts and examine the boundary between implicit framing effects and explicit prompt manipulation. Specifically, we evaluate how susceptible LLMs are to direct and suggestive prompts that encourage models to support or challenge particular positions. We evaluate six LLMs using 160 distinct prompts spanning ten topics across opinion-based and factual domains. The prompts systematically vary in prompting strategy, support versus challenge instructions, prompt polarity, users' expressed beliefs, and topic domain, spanning both opinion-based and factual questions. Our results show that LLMs systematically adapt their responses to align with prompt framing, even in factual contexts. This suggests that prompt framing can outweigh factual consistency in model responses. Overall, our findings delineate the extent and boundaries of LLM manipulability. Furthermore, the results imply that LLMs can reinforce subtle user biases and are susceptible to explicit prompt manipulation even in domains where responses should remain factually stable.
