大型语言模型的风险画像与调控
Risk Profiling and Modulation for LLMs
AI总结：
本研究提出新流程，利用效用理论模型比较不同训练阶段LLM的风险画像，发现后训练能最稳定有效地调控风险偏好，为行为对齐和风险感知设计奠定基础。
摘要：
大型语言模型（LLMs）越来越多地被用于不确定性条件下的决策任务；然而，它们的风险画像以及提示方法和对齐方法如何影响这些画像，仍未得到充分探索。现有研究主要考察了人格提示或多智能体交互，留下了关于后训练如何影响LLMs风险行为的问题。在这项工作中，我们提出了一种新的流程，用于引出、引导和调控LLMs的风险画像，借鉴了行为经济学和金融学的工具。利用效用理论模型，我们比较了预训练、指令微调和RLHF对齐的LLMs，发现虽然指令微调模型表现出与某些标准效用公式一致的行为，但预训练和RLHF对齐的模型更偏离任何拟合的效用模型。我们进一步评估了调控策略，包括提示工程、上下文学习和后训练，并表明后训练提供了最稳定和有效的风险偏好调控。我们的发现为不同类别和阶段的LLMs的风险画像提供了见解，并展示了后训练如何调控这些画像，为行为对齐和风险感知LLM设计的未来研究奠定了基础。
英文摘要：
Large language models (LLMs) are increasingly used for decision-making tasks under uncertainty; however, their risk profiles and how they are influenced by prompting and alignment methods remain underexplored. Existing studies have primarily examined personality prompting or multi-agent interactions, leaving open the question of how post-training influences the risk behavior of LLMs. In this work, we propose a new pipeline for eliciting, steering, and modulating LLMs' risk profiles, drawing on tools from behavioral economics and finance. Using utility-theoretic models, we compare pre-trained, instruction-tuned, and RLHF-aligned LLMs, and find that while instruction-tuned models exhibit behaviors consistent with some standard utility formulations, pre-trained and RLHF-aligned models deviate more from any utility models fitted. We further evaluate modulation strategies, including prompt engineering, in-context learning, and post-training, and show that post-training provides the most stable and effective modulation of risk preference. Our findings provide insights into the risk profiles of different classes and stages of LLMs and demonstrate how post-training modulates these profiles, laying the groundwork for future research on behavioral alignment and risk-aware LLM design.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
