arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1506.02188cs.AImath.OC
收藏

风险敏感鲁棒决策的CVaR优化方法

Risk-Sensitive and Robust Decision-Making: a CVaR Optimization Approach

Stanford University(斯坦福大学);The Technion - Israel Institute of Technology(以色列理工学院)

Yinlam Chow, Aviv Tamar, Shie Mannor, Marco Pavone

摘要:

在本文中,我们解决的问题,在马尔可夫决策过程(MDP)的框架内,风险和建模误差考虑的决策。我们的方法是最小化风险敏感的条件风险价值(CVaR)目标,而不是标准的风险中性预期。我们把这样的问题称为CVaRMDP。我们的第一个贡献是表明,CVaR的目标,除了捕捉风险敏感性,有一个替代的解释,在最坏情况下的建模误差的预期成本,对于给定的误差预算。这一结果,这是独立的利益,激励CVaRMDP作为一个统一的框架,风险敏感和强大的决策。我们的第二个贡献是提出了一个近似值迭代算法CVaRMDP和分析其收敛速度。据我们所知,这是CVaRMDP的第一个解决方案算法,享有误差保证。最后,我们提出的数值实验结果,证实了我们的理论研究结果,并显示我们的方法的实用性。

英文摘要:

In this paper we address the problem of decision making within a Markov decision process (MDP) framework where risk and modeling errors are taken into account. Our approach is to minimize a risk-sensitive conditional-value-at-risk (CVaR) objective, as opposed to a standard risk-neutral expectation. We refer to such problem as CVaR MDP. Our first contribution is to show that a CVaR objective, besides capturing risk sensitivity, has an alternative interpretation as expected cost under worst-case modeling errors, for a given error budget. This result, which is of independent interest, motivates CVaR MDPs as a unifying framework for risk-sensitive and robust decision making. Our second contribution is to present an approximate value-iteration algorithm for CVaR MDPs and analyze its convergence rate. To our knowledge, this is the first solution algorithm for CVaR MDPs that enjoys error guarantees. Finally, we present results from numerical experiments that corroborate our theoretical findings and show the practicality of our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑