arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2208.04433cs.GT
收藏

学习Agent的节点预测

Peer Prediction for Learning Agents

Shi Feng, Fang-Yi Yu, Yiling Chen

摘要:

对等预测是指当无法直接验证所获得的信息时,从人类代理中获取信息的机制的集合。它们被设计成一个博弈论均衡,每个人都如实地透露自己的私人信息。这个结果在代理是贝叶斯的假设下成立,并且他们每个人都在所有任务中采用固定的策略。然而,在许多领域中观察到人类代理在顺序设置中表现出学习行为。在本文中,我们探讨了动态的顺序同行预测机制时,参与者是学习代理。我们首先表明,没有后悔的概念,单独的代理的学习算法不能保证收敛到真实的战略。然后,我们专注于一个家庭的学习算法的策略更新只依赖于代理的累积奖励,并证明代理的策略在流行的相关协议(CA)机制收敛到真实的报告时,他们使用这个家庭的算法。这个算法族不一定是无遗憾的,但包括几个熟悉的无遗憾学习算法(例如乘法权重更新和跟随扰动领导者)作为特例。对该算法族中的几种算法以及该算法族之外的$ε$ -贪心算法的仿真结果表明,该算法在CA机制中收敛于真实策略。

英文摘要:

Peer prediction refers to a collection of mechanisms for eliciting information from human agents when direct verification of the obtained information is unavailable. They are designed to have a game-theoretic equilibrium where everyone reveals their private information truthfully. This result holds under the assumption that agents are Bayesian and they each adopt a fixed strategy across all tasks. Human agents however are observed in many domains to exhibit learning behavior in sequential settings. In this paper, we explore the dynamics of sequential peer prediction mechanisms when participants are learning agents. We first show that the notion of no regret alone for the agents' learning algorithms cannot guarantee convergence to the truthful strategy. We then focus on a family of learning algorithms where strategy updates only depend on agents' cumulative rewards and prove that agents' strategies in the popular Correlated Agreement (CA) mechanism converge to truthful reporting when they use algorithms from this family. This family of algorithms is not necessarily no-regret, but includes several familiar no-regret learning algorithms (e.g multiplicative weight update and Follow the Perturbed Leader) as special cases. Simulation of several algorithms in this family as well as the $ε$-greedy algorithm, which is outside of this family, shows convergence to the truthful strategy in the CA mechanism.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑