arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.02654cs.LGcs.AI
收藏

智能代理和机制的学习算法

Learning Algorithms for Intelligent Agents and Mechanisms

Princeton University(普林斯顿大学)

Jad Rahme

摘要:

在这篇论文中,我们在两个不同的背景下研究了最优决策的学习算法,第一部分是强化学习,第二部分是拍卖设计。强化学习(RL)是机器学习的一个领域,它关注的是智能体在环境中应该如何行动,以便随着时间的推移最大化其累积奖励。在第2章中,受统计物理学的启发,我们开发了一种新的强化学习(RL)方法,该方法不仅可以学习具有增强的理想属性的最优策略,还可以为最大熵RL提供新的启发。在第3章中,我们使用贝叶斯观点来解决RL中的泛化问题。我们发现,不完善的知识的环境动态有效地把一个完全观察到的马尔可夫决策过程(MDP)到一个部分观察到的MDP(POMDP),我们称之为认知POMDP。根据这一观察,我们开发了一种新的策略学习算法LEEP,它具有改进的泛化性能。设计一个激励相容的,个体理性的拍卖,使收益最大化是一个具有挑战性和棘手的问题。最近,已经提出了基于深度学习的方法来从数据中学习最优拍卖。虽然这种方法是成功的,但也有一些局限性,包括样本效率低,缺乏对新拍卖的推广,以及训练困难。在第四章中,我们构造了一个对称性保持的神经网络结构,EquivariantNet,它适用于匿名拍卖。EquivariantNet不仅样本效率更高,而且还能够学习拍卖规则,这些规则可以很好地推广到其他设置。在第5章中,我们提出了一个新的公式的拍卖学习问题作为一个两个玩家的游戏。由此产生的学习算法,ALGNet,更容易训练,更可靠,更适合于非平稳设置。

英文摘要:

In this thesis, we research learning algorithms for optimal decision making in two different contexts, Reinforcement Learning in Part I and Auction Design in Part II. Reinforcement learning (RL) is an area of machine learning that is concerned with how an agent should act in an environment in order to maximize its cumulative reward over time. In Chapter 2, inspired by statistical physics, we develop a novel approach to Reinforcement Learning (RL) that not only learns optimal policies with enhanced desirable properties but also sheds new light on maximum entropy RL. In Chapter 3, we tackle the generalization problem in RL using a Bayesian perspective. We show that imperfect knowledge of the environments dynamics effectively turn a fully-observed Markov Decision Process (MDP) into a Partially Observed MDP (POMDP) that we call the Epistemic POMDP. Informed by this observation, we develop a new policy learning algorithm LEEP which has improved generalization properties. Designing an incentive compatible, individually rational auction that maximizes revenue is a challenging and intractable problem. Recently, deep learning based approaches have been proposed to learn optimal auctions from data. While successful, this approach suffers from a few limitations, including sample inefficiency, lack of generalization to new auctions, and training difficulties. In Chapter 4, we construct a symmetry preserving neural network architecture, EquivariantNet, suitable for anonymous auctions. EquivariantNet is not only more sample efficient but is also able to learn auction rules that generalize well to other settings. In Chapter 5, we propose a novel formulation of the auction learning problem as a two player game. The resulting learning algorithm, ALGNet, is easier to train, more reliable and better suited for non stationary settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑