arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2604.08588cs.LGcs.AI
收藏

行动或升级?基于语言模型评估自动化中的升级行为

Act or Escalate? Evaluating Escalation Behavior in Automation with Language Models

Harvard Business School(哈佛商学院);Johns Hopkins Carey Business School(约翰霍普金斯大学凯瑞商学院)

Matthew DosSantos DiSorbo, Harang Ju

AI总结:

本文研究了语言模型在自动化决策中如何平衡行动与升级,发现模型在成本权衡上的隐含阈值存在显著差异,且自我评估存在偏差,通过干预测试发现链式思维训练能产生稳健的策略。

摘要:

有效的自动化依赖于决定何时行动、何时升级。我们将此建模为不确定性下的决策问题:LLM形成预测,估计其正确性的概率,并比较行动和升级的预期成本。在五个领域(需求预测、内容推荐、内容审核、贷款审批和自动驾驶)和多个模型家族中,我们发现模型在权衡这些成本时的隐含阈值存在显著差异。这些阈值变化大,不受架构或规模影响,而自我评估在模型特定的方式下存在偏差。我们随后测试了针对此决策过程的干预措施,通过调整成本比率、提供准确性信号和训练模型遵循预期的升级规则。提示主要帮助推理模型。在链式思维目标上进行SFT产生最稳健的策略,这些策略能跨数据集、成本比率、提示框架和未见领域泛化。这些结果表明,升级行为是模型特定的属性,应在部署前进行表征,并且稳健对齐受益于训练模型显式推理不确定性与决策成本。

英文摘要:

Effective automation hinges on deciding when to act and when to escalate. We model this as a decision under uncertainty: an LLM forms a prediction, estimates its probability of being correct, and compares the expected costs of acting and escalating. Using this framework across five domains of recorded human decisions-demand forecasting, content recommendation, content moderation, loan approval, and autonomous driving-and across multiple model families, we find marked differences in the implicit thresholds models use to trade off these costs. These thresholds vary substantially and are not predicted by architecture or scale, while self-estimates are miscalibrated in model-specific ways. We then test interventions that target this decision process by varying cost ratios, providing accuracy signals, and training models to follow the desired escalation rule. Prompting helps mainly for reasoning models. SFT on chain-of-thought targets yields the most robust policies, which generalize across datasets, cost ratios, prompt framings, and held-out domains. These results suggest that escalation behavior is a model-specific property that should be characterized before deployment, and that robust alignment benefits from training models to reason explicitly about uncertainty and decision costs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑