演化不确定性下的风险量化：面向安全序贯决策的信念依赖鲁棒性
Quantifying Risk Under Evolving Uncertainty: Belief-Dependent Robustness for Safe Sequential Decision Making
浏览论文内容
中文总结 AI 辅助
本文提出RATTL算法，将智能体谨慎程度与认知不确定性绑定，证明其规划问题适定且满足安全三明治性质，可保障LLM等智能体在不确定性下的运行时安全。
中文摘要 AI 辅助
智能体在学习环境过程中应保持多大程度的谨慎？本文提出RATTL（风险对抗性总奖励学习），其将谨慎程度与认知不确定性绑定：智能体对未知动力学持有贝叶斯后验分布，并针对Wasserstein模糊集进行规划，该模糊集的半径是此后验的单调函数。该半径随证据收缩，因此行为在最坏情况鲁棒性与风险中性总奖励最大化之间连续插值。该设计遵循熵值风险（Entropic Value-at-Risk）背后的对偶性，将风险水平的选择转化为模糊半径的选择。本文证明，在暂态性和紧性条件下，所得规划问题适定，并证明了安全三明治性质：RATTL价值介于未知情鲁棒价值与全知最优价值之间，且该间隙随后验集中而消失。在经典二元危险实例中，该准则简化为后验熵设定水平下的条件风险价值（Conditional Value-at-Risk）。一个示例表明，智能体将有效行动推迟至明确识别阈值。RATTL旨在为包括基于大语言模型（LLM）系统在内的、在不确定性下行动的智能体提供运行时安全保障。
英文摘要
How cautious should an agent be while it is still learning its environment? We propose RATTL (Risk-Adversarial Total-Reward Learning), which ties caution to epistemic uncertainty: the agent holds a Bayesian posterior over unknown dynamics and plans against a Wasserstein ambiguity set whose radius is a monotone function of that posterior. The radius contracts with evidence, so behaviour interpolates continuously between worst-case robustness and risk-neutral total-reward maximization. The design follows the duality underlying the Entropic Value-at-Risk, which converts the choice of a risk level into the choice of an ambiguity radius. We show the resulting planning problem is well posed under transience and compactness conditions, and prove a Safety Sandwich: the RATTL value lies between the uninformed robust value and the full- knowledge optimum, with a gap that vanishes as the posterior concentrates. In a canonical binary-hazard instance, the induced criterion reduces to Conditional Value-at-Risk at a level set by the posterior entropy. A worked example shows the agent deferring the efficient action until a sharp identification threshold. RATTL targets runtime safety for agents, including LLM-based systems, acting under uncertainty.
发表机构
- Technical University of Munich（慕尼黑工业大学）
- Masaryk University（马萨里克大学）
机构由 AI 辅助整理，请以论文原文为准。
