我们提出了一种协作贝叶斯优化问题，用于优化含两个变量的黑盒函数：两个智能体共同选择查询函数的点，但各自仅能控制其中一个变量。该设定受人机团队协作启发，即AI助手帮助人类用户解决问题，在此最简场景下为协作优化任务。我们将解决方案建模为序列决策问题：我方控制的智能体将用户建模为具备函数先验知识的计算理性智能体。研究表明，只要用户避免过度探索，对查询进行策略性规划就能更好地识别函数的全局最大值。该规划通过贝叶斯自适应蒙特卡洛规划（Bayes Adaptive Monte Carlo planning）实现，同时为智能体配备用户模型，该模型可解释保守的信念更新过程以及待查询点的探索性采样行为。
英文摘要：
We introduce a cooperative Bayesian optimization problem for optimizing black-box functions of two variables where two agents choose together at which points to query the function but have only control over one variable each. This setting is inspired by human-AI teamwork, where an AI-assistant helps its human user solve a problem, in this simplest case, collaborative optimization. We formulate the solution as sequential decision-making, where the agent we control models the user as a computationally rational agent with prior knowledge about the function. We show that strategic planning of the queries enables better identification of the global maximum of the function as long as the user avoids excessive exploration. This planning is made possible by using Bayes Adaptive Monte Carlo planning and by endowing the agent with a user model that accounts for conservative belief updates and exploratory sampling of the points to query.
University of Alberta（阿尔伯塔大学）；McGill University（麦吉尔大学）；MILA（蒙特利尔学习算法研究所）；DeepMind（深度思维）；Samsung（三星电子）
Guided exploration with expert demonstrations improves data efficiency for reinforcement learning, but current algorithms often overuse expert information. We propose a novel algorithm to speed up Q-learning with the help of a limited amount of imperfect expert demonstrations. The algorithm avoids excessive reliance on expert data by relaxing the optimal expert assumption and gradually reducing the usage of uninformative expert data. Experimentally, we evaluate our approach on a sparse-reward chain environment and six more complicated Atari games with delayed rewards. With the proposed methods…
Bayesian Optimization for Non-Cooperative Game-Based Radio Resource Management
School of Information Engineering, Wuhan University of Technology, Wuhan, China（信息工程学院，武汉理工大学，武汉）；Department of Engineering, King's College London, London, UK（工程系，伦敦国王学院，伦敦）；School of Electronic Information and Communications, Huazhong University of Science（电子信息与通信学院，华中科技大学）
Arizona State University（亚利桑那州立大学）；Boston University（波士顿大学）；University of Illinois（伊利诺伊大学）
We study the problem of cooperative inference where a group of agents interact over a network and seek to estimate a joint parameter that best explains a set of observations. Agents do not know the network topology or the observations of other agents. We explore a variational interpretation of the Bayesian posterior density, and its relation to the stochastic mirror descent algorithm, to propose a new distributed learning algorithm. We show that, under appropriate assumptions, the beliefs generated by the proposed algorithm concentrate around the true parameter exponentially fast. We provide e…
Scalable Decision Making for Games of Imperfect Information
Carnegie Mellon University（卡内基梅隆大学）；NYU Tandon School of Engineering（纽约大学坦登工程学院）；Stanford University（斯坦福大学）；Massachusetts Institute of Technology（麻省理工学院）
Macau University of Science and Technology（澳门科技大学）
The Distributed Cooperative Modeling System (DCMS) solves complex decision problems involving a lot of participants with different viewpoints by network based distributed modeling and multi-template aggregation. This thesis aims at extending the system with support for dynamic decision making process. First, the thesis presents a discussion of characteristics and optimal policy finding Markov Decision Process as well as a brief introduction to dynamic Bayesian decision network, which is inherently equal to MDP. After that, discussion and implementation of prediction in Markov process for both …
Overcoming the Machine Penalty with Imperfectly Fair AI Agents
School of Cybersecurity, and School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, China（网络安全学院和人工智能学院，光学电子研究所（iOPEN），西北工业大学，中国）；Faculty of Engineering Sciences, Kyushu University, Japan（工程科学学院，九州大学，日本）；School of Computing, Engineering and Digital Technologies, Teesside University, United Kingdom（计算、工程与数字技术学院，泰赛德大学，英国）；Computer Science, Science Division, New York University Abu Dhabi, UAE（计算机科学，科学系，纽约大学阿布扎比分校，阿联酋）；School of Statistics and Mathematics, Yunnan University of Finance and Economics, China（统计与数学学院，云南财经大学，中国）；Shanghai Artificial Intelligence Laboratory, China（上海人工智能实验室，中国）
Princeton University（普林斯顿大学）；Massachusetts Institute of Technology（麻省理工学院）
The cooperative bandit problem is increasingly becoming relevant due to its applications in large-scale decision-making. However, most research for this problem focuses exclusively on the setting with perfect communication, whereas in most real-world distributed settings, communication is often over stochastic networks, with arbitrary corruptions and delays. In this paper, we study cooperative bandit learning under three typical real-world communication scenarios, namely, (a) message-passing over stochastic time-varying networks, (b) instantaneous reward-sharing over a network with random dela…
Cooperation is often implicitly assumed when learning from other agents. Cooperation implies that the agent selecting the data, and the agent learning from the data, have the same goal, that the learner infer the intended hypothesis. Recent models in human and machine learning have demonstrated the possibility of cooperation. We seek foundational theoretical results for cooperative inference by Bayesian agents through sequential data. We develop novel approaches analyzing consistency, rate of convergence and stability of Sequential Cooperative Bayesian Inference (SCBI). Our analysis of the eff…
For tasks where the dynamics of multiple agents are physically coupled, e.g., in cooperative manipulation, the coordination between the individual agents becomes crucial, which requires exact knowledge of the interaction dynamics. This problem is typically addressed using centralized estimators, which can negatively impact the flexibility and robustness of the overall system. To overcome this shortcoming, we propose a novel distributed learning framework for the exemplary task of cooperative manipulation using Bayesian principles. Using only local state information each agent obtains an estima…
Harvard University（哈佛大学）；KU Leuven（鲁汶大学）；IMT School for Advanced Studies（IMT高等研究院）
This paper introduces an innovative Bayesian machine learning algorithm to draw interpretable inference on heterogeneous causal effects in the presence of imperfect compliance (e.g., under an irregular assignment mechanism). We show, through Monte Carlo simulations, that the proposed Bayesian Causal Forest with Instrumental Variable (BCF-IV) methodology outperforms other machine learning techniques tailored for causal inference in discovering and estimating the heterogeneous causal effects while controlling for the familywise error rate (or - less stringently - for the false discovery rate) at…
We introduce a computational efficient data-driven framework suitable for quantifying the uncertainty in physical parameters and model formulation of computer models, represented by differential equations. We construct physics-informed priors, which are multi-output GP priors that encode the model's structure in the covariance function. This is extended into a fully Bayesian framework that quantifies the uncertainty of physical parameters and model predictions. Since physical models often are imperfect descriptions of the real process, we allow the model to deviate from the observed data by co…
Cooperative Variance Estimation and Bayesian Neural Networks for Disentangling Aleatoric and Epistemic Uncertainties
Faculty of Mechanical Engineering, Delft University of Technology, Mekelweg 2, Delft, 2628 CD, The Netherlands（代尔夫特理工大学机械工程学院）；School of Engineering, Brown University, 184 Hope St., Providence, RI 02912, USA（布朗大学工程学院）
MSCoD: An Enhanced Bayesian Updating Framework with Multi-Scale Information Bottleneck and Cooperative Attention for Structure-Based Drug Design
Guangxi Key Lab of Human-machine Interaction and Intelligent Decision（广西人机交互与智能决策重点实验室）；Nanning Normal University（南宁师范学院）；College of Big Data and Software Engineering（大数据与软件工程学院）；Zhejiang Wanli University（浙江万里大学）
Reinforcement Learning with Imperfect Transition Predictions: A Bellman-Jensen Approach
Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；Edwardson School of Industrial Engineering, Purdue University（普渡大学爱德华森工业工程学院）；School of Data Science, The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）数据科学学院）；School of Science and Engineering, The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）理工学院）；Computing & Mathematical Sciences, Caltech（加州理工学院计算与数学科学系）
To learn good joint policies for multi-agent collaboration with imperfect information remains a fundamental challenge. While for two-player zero-sum games, coordinate-ascent approaches (optimizing one agent's policy at a time, e.g., self-play) work with guarantees, in multi-agent cooperative setting they often converge to sub-optimal Nash equilibrium. On the other hand, directly modeling joint policy changes in imperfect information game is nontrivial due to complicated interplay of policies (e.g., upstream updates affect downstream state reachability). In this paper, we show global changes of…
