基于偏好的蒙特卡罗树搜索
Preference-Based Monte Carlo Tree Search
摘要：
蒙特卡罗树搜索（MCTS）是求解序列任意时刻问题的常用算法.然而，它取决于数字反馈信号，这可能很难定义。实时MCTS是一种变体，它可能很少遇到具有明确外在奖励的状态。为了处理这种情况，实验者必须以启发式的形式提供额外的数字反馈信号，这在本质上指导代理。最近的研究表明，在不同的领域，基本结构是有序的，而不是数字的。因此，错误和有偏见的解释是不可避免的，特别是在这些领域。在本文中，我们提出了一个MCTS的变种，它只依赖于定性反馈，因此开辟了新的应用MCTS。我们还发现，将绝对的序数反馈可能是有益的迹象。使用谜题域，我们表明我们基于偏好的MCTS变体（仅接收定性反馈）能够达到与获得定量反馈的常规MCTS基线相当的性能水平。
英文摘要：
Monte Carlo tree search (MCTS) is a popular choice for solving sequential anytime problems. However, it depends on a numeric feedback signal, which can be difficult to define. Real-time MCTS is a variant which may only rarely encounter states with an explicit, extrinsic reward. To deal with such cases, the experimenter has to supply an additional numeric feedback signal in the form of a heuristic, which intrinsically guides the agent. Recent work has shown evidence that in different areas the underlying structure is ordinal and not numerical. Hence erroneous and biased heuristics are inevitable, especially in such domains. In this paper, we propose a MCTS variant which only depends on qualitative feedback, and therefore opens up new applications for MCTS. We also find indications that translating absolute into ordinal feedback may be beneficial. Using a puzzle domain, we show that our preference-based MCTS variant, wich only receives qualitative feedback, is able to reach a performance level comparable to a regular MCTS baseline, which obtains quantitative feedback.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
