Risk is 6 player game with significant randomness and a large game-tree complexity which poses a challenge to creating an agent to play the game effectively. Previous AIs focus on creating high-level handcrafted features determine agent decision making. In this project, I create D.A.D, A Risk agent using temporal difference reinforcement learning to train a Deep Neural Network including a Graph Convolutional Network to evaluate player positions. This is used in a game-tree to select optimal moves. This allows minimal handcrafting of knowledge into the AI, assuring input features are as low-level as possible to allow the network to extract useful and sophisticated features itself, even with the network starting from a random initialisation. I also tackle the issue of non-determinism in Risk by introducing a new method of interpreting attack moves necessary for the search. The result is an AI which wins 35% of the time versus 5 of best inbuilt AIs in Lux Delux, a Risk variant.
The game of Go has a long history in East Asian countries, but the field of Computer Go has yet to catch up to humans until the past couple of years. While the rules of Go are simple, the strategy and combinatorics of the game are immensely complex. Even within the past couple of years, new programs that rely on neural networks to evaluate board positions still explore many orders of magnitude more board positions per second than a professional can. We attempt to mimic human intuition in the game by creating a convolutional neural policy network which, without any sort of tree search, should p…
StarCraft, one of the most popular real-time strategy games, is a compelling environment for artificial intelligence research for both micro-level unit control and macro-level strategic decision making. In this study, we address an eminent problem concerning macro-level decision making, known as the 'fog-of-war', which rises naturally from the fact that information regarding the opponent's state is always provided in the incomplete form. For intelligent agents to play like human players, it is obvious that making accurate predictions of the opponent's status under incomplete information will i…
University College London（伦敦大学学院）；Huawei Technologies（华为技术有限公司）；Peking University（北京大学）
In order for agents in multi-agent systems (MAS) to be safe, they need to take into account the risks posed by the actions of other agents. However, the dominant paradigm in game theory (GT) assumes that agents are not affected by risk from other agents and only strive to maximise their expected utility. For example, in hybrid human-AI driving systems, it is necessary to limit large deviations in reward resulting from car crashes. Although there are equilibrium concepts in game theory that take into account risk aversion, they either assume that agents are risk-neutral with respect to the unce…
In stochastic games with incomplete information, the uncertainty is evoked by the lack of knowledge about a player's own and the other players' types, i.e. the utility function and the policy space, and also the inherent stochasticity of different players' interactions. In existing literature, the risk in stochastic games has been studied in terms of the inherent uncertainty evoked by the variability of transitions and actions. In this work, we instead focus on the risk associated with the \textit{uncertainty over types}. We contrast this with the multi-agent reinforcement learning framework w…
KTH Royal Institute of Technology（KTH皇家理工学院）；Digital Futures（数字未来中心）；Duke University（杜克大学）；Air Force Research Laboratory（美国空军研究实验室）
We consider risk-averse learning in repeated unknown games where the goal of the agents is to minimize their individual risk of incurring significantly high cost. Specifically, the agents use the conditional value at risk (CVaR) as a risk measure and rely on bandit feedback in the form of the cost values of the selected actions at every episode to estimate their CVaR values and update their actions. A major challenge in using bandit feedback to estimate CVaR is that the agents can only access their own cost values, which, however, depend on the actions of all agents. To address this challenge,…
KTH Royal Institute of Technology（KTH皇家理工学院）；Duke University（杜克大学）
We consider an online stochastic game with risk-averse agents whose goal is to learn optimal decisions that minimize the risk of incurring significantly high costs. Specifically, we use the Conditional Value at Risk (CVaR) as a risk measure that the agents can estimate using bandit feedback in the form of the cost values of only their selected actions. Since the distributions of the cost functions depend on the actions of all agents that are generally unobservable, they are themselves unknown and, therefore, the CVaR values of the costs are difficult to compute. To address this challenge, we p…
Learning in Multi-Objective Public Goods Games with Non-Linear Utilities
University of Groningen（格罗宁根大学）；University of Amsterdam（阿姆斯特丹大学）；University of Galway（戈尔韦大学）；Utrecht University（乌得勒支大学）；Vrije Universiteit Brussels（布鲁塞尔自由大学）
The Automated but Risky Game: Modeling and Benchmarking Agent-to-Agent Negotiations and Transactions in Consumer Markets
Stanford University（斯坦福大学）；University of Toronto（多伦多大学）；Google DeepMind（谷歌DeepMind）；University of Southern California（南加州大学）；Massachusetts Institute of Technology（麻省理工学院）
The evaluation function for imperfect information games is always hard to define but owns a significant impact on the playing strength of a program. Deep learning has made great achievements these years, and already exceeded the top human players' level even in the game of Go. In this paper, we introduce a new data model to represent the available imperfect information on the game table, and construct a well-designed convolutional neural network for game record training. We choose the accuracy of tile discarding which is also called as the agreement rate as the benchmark for this study. Our ac…
The report suggests the concept of risk, outlining two mathematical structures necessary for risk genesis: the set of outcomes and, in a general case, partial order of preference on it. It is shown that this minimum partial order should constitute the structure of a semilattice. In some cases, there should be a system of semilattices nested in a certain way. On this basis, the classification of risk theory tasks is given in the context of specialization of mathematical knowledge. In other words, we are talking about the development of a new rela-tional risk theory. The problem of political dec…
Risk Analysis in Customer Relationship Management via Quantile Region Convolutional Neural Network-Long Short-Term Memory and Cross-Attention Mechanism
Longyan University（龙岩学院）；National Central University（中央大学）；University of Portsmouth（朴茨茅斯大学）；Institute of Semiconductors, Chinese Academy of Sciences（中国科学院半导体研究所）；Northeastern University（东北大学）
Autonomous systems can substantially enhance a human's efficiency and effectiveness in complex environments. Machines, however, are often unable to observe the preferences of the humans that they serve. Despite the fact that the human's and machine's objectives are aligned, asymmetric information, along with heterogeneous sensitivities to risk by the human and machine, make their joint optimization process a game with strategic interactions. We propose a framework based on risk-sensitive dynamic games; the human seeks to optimize her risk-sensitive criterion according to her true preferences, …
Mastering the game of Go has remained a long standing challenge to the field of AI. Modern computer Go systems rely on processing millions of possible future positions to play well, but intuitively a stronger and more 'humanlike' way to play the game would be to rely on pattern recognition abilities rather then brute force computation. Following this sentiment, we train deep convolutional neural networks to play Go by training them to predict the moves made by expert Go players. To solve this problem we introduce a number of novel techniques, including a method of tying weights in the network …
GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory
ETH Zürich（苏黎世联邦理工学院）；Berea College（贝雷学院）；University of Toronto（多伦多大学）；Vector Institute（向量研究所）；Max Planck Institute for Intelligent Systems, Tübingen, Germany（图宾根德国智能系统马克斯·普朗克研究所）
