期望标化收益占优：一种新的多目标决策解概念
Expected Scalarised Returns Dominance: A New Solution Concept for Multi-Objective Decision Making
摘要：
在许多现实场景中，用户的效用来自策略的单次执行。在这种情况下，为了应用多目标强化学习，必须优化收益的预期效用。存在用户对目标的偏好（也称为效用函数）未知或难以指定的各种场景。在这种情况下，必须学习一组最优策略。然而，多目标强化学习社区在很大程度上忽视了必须最大化预期效用的设置，因此，尚未定义一组最佳解决方案。在本文中，我们提出了一阶随机优势作为一个标准来建立解决方案集，以最大限度地提高预期效用，以解决这一挑战。我们还提出了一个新的优势准则，称为预期标化收益（ESR）的优势，扩展一阶随机优势，让一组最优的政策，在实践中学习。然后，我们定义了一个名为ESR集的新解决方案概念，它是一组以ESR为主导的策略。最后，我们定义了一种新的多目标分布表格强化学习（MOT-DRL）算法来学习多目标多臂强盗环境中的ESR集。
英文摘要：
In many real-world scenarios, the utility of a user is derived from the single execution of a policy. In this case, to apply multi-objective reinforcement learning, the expected utility of the returns must be optimised. Various scenarios exist where a user's preferences over objectives (also known as the utility function) are unknown or difficult to specify. In such scenarios, a set of optimal policies must be learned. However, settings where the expected utility must be maximised have been largely overlooked by the multi-objective reinforcement learning community and, as a consequence, a set of optimal solutions has yet to be defined. In this paper we address this challenge by proposing first-order stochastic dominance as a criterion to build solution sets to maximise expected utility. We also propose a new dominance criterion, known as expected scalarised returns (ESR) dominance, that extends first-order stochastic dominance to allow a set of optimal policies to be learned in practice. We then define a new solution concept called the ESR set, which is a set of policies that are ESR dominant. Finally, we define a new multi-objective distributional tabular reinforcement learning (MOT-DRL) algorithm to learn the ESR set in a multi-objective multi-armed bandit setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
