arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1702.07694stat.MLcs.ITcs.LGmath.IT
收藏

基于主动选择的贝叶斯最优熵追踪偏好学习

Bayes-Optimal Entropy Pursuit for Active Choice-Based Preference Learning

Cornell University(康奈尔大学)

Stephen N. Pallone, Peter I. Frazier, Shane G. Henderson

摘要:

我们分析的问题,学习一个单一的用户的喜好在一个积极的学习设置,顺序和自适应查询用户在有限的时间范围内。学习是通过基于选择的查询进行的,用户在提供的备选方案的一小部分中选择她的首选方案。这些查询已被证明是一种强大而有效的方式来了解个人的偏好。我们采取参数化的方法,并通过线性分类器对用户的偏好进行建模,使用贝叶斯先验对我们当前对该分类器的知识进行编码。我们学习的速度取决于每个时间点提供的选择。在某些噪声假设下,我们证明了最大限度地减少该线性分类器的后验分布的熵的贝叶斯最优策略是贪婪策略,并且当可以从连续统中构造替代方案时,该策略实现了线性下界。此外,我们分析了一个不同的度量称为误分类错误,证明了最小化误分类错误的最优策略的性能是有界的微分熵的线性函数以下。最后,我们数值比较贪婪的熵减少政策与知识梯度政策下的一些情况下,检查他们的性能下的微分熵和误分类错误。

英文摘要:

We analyze the problem of learning a single user's preferences in an active learning setting, sequentially and adaptively querying the user over a finite time horizon. Learning is conducted via choice-based queries, where the user selects her preferred option among a small subset of offered alternatives. These queries have been shown to be a robust and efficient way to learn an individual's preferences. We take a parametric approach and model the user's preferences through a linear classifier, using a Bayesian prior to encode our current knowledge of this classifier. The rate at which we learn depends on the alternatives offered at every time epoch. Under certain noise assumptions, we show that the Bayes-optimal policy for maximally reducing entropy of the posterior distribution of this linear classifier is a greedy policy, and that this policy achieves a linear lower bound when alternatives can be constructed from the continuum. Further, we analyze a different metric called misclassification error, proving that the performance of the optimal policy that minimizes misclassification error is bounded below by a linear function of differential entropy. Lastly, we numerically compare the greedy entropy reduction policy with a knowledge gradient policy under a number of scenarios, examining their performance under both differential entropy and misclassification error.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑