Thales Research & Technology（泰雷兹研究与技术公司）；Sorbonne Université（索邦大学）；Université de Reims Champagne Ardennes（兰斯香槟-阿登大学）；Universidade Estadual do Ceará（塞阿拉州立大学）
Cédric Buron, Zahia Guessoum, Sylvain Ductor
发表信息：
The 22nd International Conference on Principles and Practice of Multi-Agent Systems (PRIMA2019), Oct 2019, Torino, Italy
This paper introduces a new negotiating agent model for automated negotiation. We focus on applications without time pressure with multidi-mensional negotiation on both continuous and discrete domains. The agent bidding strategy relies on Monte Carlo Tree Search, which is a trendy method since it has been used with success on games with high branching factor such as Go. It also exploits opponent modeling techniques thanks to Gaussian process regression and Bayesian learning. Evaluation is done by confronting the existing agents that are able to negotiate in such context: Random Walker, Tit-for-tat and Nice Tit-for-Tat. None of those agents succeeds in beating our agent. Also, the modular and adaptive nature of our approach is a huge advantage when it comes to optimize it in specific applicative contexts.
This paper introduces a new Negotiating Agent for automated negotiation on continuous domains and without considering a specified deadline. The agent bidding strategy relies on Monte Carlo Tree Search, which is a trendy method since it has been used with success on games with high branching factor such as Go. It uses two opponent modeling techniques for its bidding strategy and its utility: Gaussian process regression and Bayesian learning. Evaluation is done by confronting the existing agents that are able to negotiate in such context: Random Walker, Tit-for-tat and Nice Tit-for-Tat. None of …
Royal Holloway, University of London（伦敦大学皇家霍洛威学院）；King Saud University（沙特国王大学）
We present a novel negotiation model that allows an agent to learn how to negotiate during concurrent bilateral negotiations in unknown and dynamic e-markets. The agent uses an actor-critic architecture with model-free reinforcement learning to learn a strategy expressed as a deep neural network. We pre-train the strategy by supervision from synthetic market data, thereby decreasing the exploration time required for learning during negotiation. As a result, we can build automated agents for concurrent negotiations that can adapt to different e-market settings without the need to be pre-program…
Northeastern University（东北大学）；New York University（纽约大学）；University of Malta（马耳他大学）
This paper describes a method for generative player modeling and its application to the automatic testing of game content using archetypal player models called procedural personas. Theoretically grounded in psychological decision theory, procedural personas are implemented using a variation of Monte Carlo Tree Search (MCTS) where the node selection criteria are developed using evolutionary computation, replacing the standard UCB1 criterion of MCTS. Using these personas we demonstrate how generative player models can be applied to a varied corpus of game levels and demonstrate how different pla…
We present a novel bilateral negotiation model that allows a self-interested agent to learn how to negotiate over multiple issues in the presence of user preference uncertainty. The model relies upon interpretable strategy templates representing the tactics the agent should employ during the negotiation and learns template parameters to maximize the average utility received over multiple negotiations, thus resulting in optimal bid acceptance and generation. Our model also uses deep reinforcement learning to evaluate threshold utility values, for those tactics that require them, thereby derivin…
The Automated but Risky Game: Modeling and Benchmarking Agent-to-Agent Negotiations and Transactions in Consumer Markets
Stanford University（斯坦福大学）；University of Toronto（多伦多大学）；Google DeepMind（谷歌DeepMind）；University of Southern California（南加州大学）；Massachusetts Institute of Technology（麻省理工学院）
Despite abundant negotiation strategies in literature, the complexity of automated negotiation forbids a single strategy from being dominant against all others in different negotiation scenarios. To overcome this, one approach is to use mixture of experts, but at the same time, one problem of this method is the selection of experts, as this approach is limited by the competency of the experts selected. Another problem with most negotiation strategies is their incapability of adapting to dynamic variation of the opponent's behaviour within a single negotiation session resulting in poor performa…
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；FZI Research Center for Information Technology（FZI信息技术研究中心）
Today's automated vehicles lack the ability to cooperate implicitly with others. This work presents a Monte Carlo Tree Search (MCTS) based approach for decentralized cooperative planning using macro-actions for automated vehicles in heterogeneous environments. Based on cooperative modeling of other agents and Decoupled-UCT (a variant of MCTS), the algorithm evaluates the state-action-values of each agent in a cooperative and decentralized manner, explicitly modeling the interdependence of actions between traffic participants. Macro-actions allow for temporal extension over multiple time steps …
This paper presents a novel approach to automated playtesting for the prediction of human player behavior and experience. It has previously been demonstrated that Deep Reinforcement Learning (DRL) game-playing agents can predict both game difficulty and player engagement, operationalized as average pass and churn rates. We improve this approach by enhancing DRL with Monte Carlo Tree Search (MCTS). We also motivate an enhanced selection strategy for predictor features, based on the observation that an AI agent's best-case performance can yield stronger correlations with human data than the agen…
Monte Carlo Tree Search (MCTS) has proven to be capable of solving challenging tasks in domains such as Go, chess and Atari. Previous research has developed parallel versions of MCTS, exploiting today's multiprocessing architectures. These studies focused on versions of MCTS for the discrete case. Our work builds upon existing parallelization strategies and extends them to continuous domains. In particular, leaf parallelization and root parallelization are studied and two final selection strategies that are required to handle continuous states in root parallelization are proposed. The evaluati…
FZI Research Center for Information Technology（FZI信息技术研究中心）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）
Urban traffic scenarios often require a high degree of cooperation between traffic participants to ensure safety and efficiency. Observing the behavior of others, humans infer whether or not others are cooperating. This work aims to extend the capabilities of automated vehicles, enabling them to cooperate implicitly in heterogeneous environments. Continuous actions allow for arbitrary trajectories and hence are applicable to a much wider class of problems than existing cooperative approaches with discrete action spaces. Based on cooperative modeling of other agents, Monte Carlo Tree Search (MC…
In this work we propose an ontology to support automated negotiation in multiagent systems. The ontology can be connected with some domain-specific ontologies to facilitate the negotiation in different domains, such as Intelligent Transportation Systems (ITS), e-commerce, etc. The specific negotiation rules for each type of negotiation strategy can also be defined as part of the ontology, reducing the amount of knowledge hardcoded in the agents and ensuring the interoperability. The expressiveness of the ontology was proved in a multiagent architecture for the automatic traffic light setting a…
Arcane: An Assertion Reduction Framework through Semantic Clustering and MCTS-Guided Rule Exploring
State Key Lab of Processors, Institute of Computing Technology, CAS, Beijing, China（处理器国家重点实验室，计算技术研究所，中国科学院，北京，中国）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学，北京，中国）
University of Amsterdam（阿姆斯特丹大学）；Delft University of Technology（代尔夫特理工大学）
Understanding an opponent agent helps in negotiating with it. Existing works on understanding opponents focus on preference modeling (or estimating the opponent's utility function). An important but largely unexplored direction is recognizing an opponent's negotiation strategy, which captures the opponent's tactics, e.g., to be tough at the beginning but to concede toward the deadline. Recognizing complex, state-of-the-art, negotiation strategies is extremely challenging, and simple heuristics may not be adequate for this purpose. We propose a novel data-driven approach for recognizing an oppo…
Delft University of Technology（代尔夫特理工大学）；Leiden University（莱顿大学）
Bidding and acceptance strategies have a substantial impact on the outcome of negotiations in scenarios with linear additive and nonlinear utility functions. Over the years, it has become clear that there is no single best strategy for all negotiation settings, yet many fixed strategies are still being developed. We envision a shift in the strategy design question from: What is a good strategy?, towards: What could be a good strategy? For this purpose, we developed a method leveraging automated algorithm configuration to find the best strategies for a specific set of negotiation settings. By e…
The University of Melbourne（墨尔本大学）；Centrum Wiskunde & Informatica（荷兰国家数学与计算机科学研究所）；Utrecht University（乌得勒支大学）
This paper presents an automated peer-to-peer negotiation strategy for settling energy contracts among prosumers in a Residential Energy Cooperative considering heterogeneity prosumer preferences. The heterogeneity arises from prosumers' evaluation of energy contracts through multiple societal and environmental criteria and the prosumers' private preferences over those criteria. The prosumers engage in bilateral negotiations with peers to mutually agree on periodical energy contracts/loans consisting of the energy volume to be exchanged at that period and the return time of the exchanged energ…
In this paper, we present a new methodology that employs tester agents to automate video game testing. We introduce two types of agents -synthetic and human-like- and two distinct approaches to create them. Our agents are derived from Reinforcement Learning (RL) and Monte Carlo Tree Search (MCTS) agents, but focus on finding defects. The synthetic agent uses test goals generated from game scenarios, and these goals are further modified to examine the effects of unintended game transitions. The human-like agent uses test goals extracted by our proposed multiple greedy-policy inverse reinforceme…
REsearch Groups in Intelligent Machines（智能机器研究组）；University of Sfax（斯法克斯大学）；ENIS（斯法克斯国立工程师学院）
Feature Selection (FS) plays an important role in learning and classification tasks. The object of FS is to select the relevant and non-redundant features. Considering the huge amount number of features in real-world applications, FS methods using batch learning technique can't resolve big data problem especially when data arrive sequentially. In this paper, we propose an online feature selection system which resolves this problem. More specifically, we treat the problem of online supervised feature selection for binary classification as a decision-making problem. A philosophical vision to thi…
This paper presents an automated peer-to-peer (P2P) negotiation strategy for settling energy contracts among prosumers in a Residential Energy Cooperative (REC) considering heterogeneous prosumer preferences. The heterogeneity arises from prosumers' evaluation of energy contracts through multiple societal and environmental criteria and the prosumers' private preferences over those criteria. The prosumers engage in bilateral negotiations with peers to mutually agree on periodical energy contracts/loans that consist of an energy volume to be exchanged at that period and the return time of the ex…
Bargaining can be used to resolve mixed-motive games in multi-agent systems. Although there is an abundance of negotiation strategies implemented in automated negotiating agents, most agents are based on single fixed strategies, while it is widely acknowledged that there is no single best-performing strategy for all negotiation settings. In this paper, we focus on bargaining settings where opponents are repeatedly encountered, but the bargaining problems change. We introduce a novel method that automatically creates and deploys a portfolio of complementary negotiation strategies using a traini…
In various multi-agent negotiation settings, a negotiator's utility depends, either partially or fully, on the sum of negotiators' utilities (i.e., social welfare). While the need for effective negotiating-agent designs that take into account social welfare has been acknowledged in recent work, and even established as a category in automated negotiating agent competitions, very few designs have been proposed to date. In this paper, we present the design principles and results of an extensive evaluation of agent HerbT+, a negotiating agent aiming to maximize a linear tradeoff between individual…
Cloud computing allows subscription based access to computing. It also allows storage services over Internet. Automated Negotiation is becoming an emerging, and important area in the field of Multi Agent Systems in ECommerce. Multi Agent based negotiation system is necessary to increase the efficiency of E-negotiation process. Cloud computing provides security and privacy to the user data and low maintenance costs. We propose a Negotiation system using cloud. In this system, all product information and multiple agent details are stored on cloud. Both parties select their agents through cloud f…
[COMP25] The Automated Negotiating Agents Competition (ANAC) 2025 Challenges and Results
Delft University of Technology（代尔夫特理工大学）；Centrum Wiskunde & Informatica (CWI)（数学与计算机科学中心）；Eindhoven University of Technology（埃因霍温理工大学）；Utrecht University（乌得勒支大学）；Tokyo University of Agriculture and Technology（东京农工大学）；National Institute for Advanced Industrial Science and Technology（产业技术综合研究所）；Leiden University（莱顿大学）；NEC Corporation（日本电气株式会社）
WirelessAgent++: Automated Agentic Workflow Design and Benchmarking for Wireless Networks
College of Electronics and Information Engineering, Shenzhen University（深圳大学电子与信息工程学院）；Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（香港理工大学电子与计算机工程系）
With the proliferation of web technologies it becomes more and more important to make the traditional negotiation pricing mechanism automated and intelligent. The behaviour of software agents which negotiate on behalf of humans is determined by their tactics in the form of decision functions. Prediction of partners behaviour in negotiation has been an active research direction in recent years as it will improve the utility gain for the adaptive negotiation agent and also achieve the agreement much quicker or look after much higher benefits. In this paper we review the various negotiation metho…
With the evolution of cloud computing, there has been a rise of large enterprises extending their infrastructure and workloads into the public cloud. This paper proposes a full-fledged framework for a Belief-Desire-Intention (BDI) multi-agent-based cloud marketplace system for cloud resources. Each party in the cloud marketplace system supports a BDI agent for autonomous decision making and negotiation to facilitate automated buying and selling of resources. Additionally, multiple BDI agents from an enterprise competing for the same cloud resource can consult with each other via Master Negotia…
Artificially intelligent agents equipped with strategic skills that can negotiate during their interactions with other natural or artificial agents are still underdeveloped. This paper describes a successful application of Deep Reinforcement Learning (DRL) for training intelligent agents with strategic conversational skills, in a situated dialogue setting. Previous studies have modelled the behaviour of strategic agents using supervised learning and traditional reinforcement learning techniques, the latter using tabular representations or learning with linear function approximation. In this st…
With the advent of evolution of cloud computing, large organizations have been scaling the on-premise IT infrastructure to the cloud. Although this being a popular practice, it lacks comprehensive efforts to study the aspects of automated negotiation of resources among cloud customers and providers. This paper proposes a full-fledged framework for the multi-party, multi-issue negotiation system for cloud resources. It introduces a robust cloud marketplace system to buy and sell cloud resources. The Belief-Desire-Intention (BDI) model-based cloud customer and provider agents concurrently negoti…
In this paper, we present the strategy of Agent Madoff, which is a heuristic-based negotiation agent that won 2nd place at the Automated Negotiating Agents Competition (ANAC 2017). Agent Madoff is implemented to play the game Diplomacy, which is a strategic board game that mimics the situation during World War I. Each player represents a major European power which has to negotiate with other forces and win possession of a majority supply centers on the map. We propose a design architecture which consists of 3 components: heuristic module, acceptance strategy and bidding strategy. The heuristic…
Kyriba；Université Pierre et Marie Curie（皮埃尔和玛丽·居里大学）；Université de Reims Champagne Ardennes（兰斯香槟阿登大学）
Opponent modeling consists in modeling the strategy or preferences of an agent thanks to the data it provides. In the context of automated negotiation and with machine learning, it can result in an advantage so overwhelming that it may restrain some casual agents to be part of the bargaining process. We qualify as "curious" an agent driven by the desire of negotiating in order to collect information and improve its opponent model. However, neither curiosity-based rational-ity nor curiosity-robust protocol have been studied in automatic negotiation. In this paper, we rely on mechanism design to…
AgentLTV: An Agent-Based Unified Search-and-Evolution Framework for Automated Lifetime Value Prediction
Sichuan University（四川大学）；Harbin Institute of Technology（哈尔滨工业大学）；Sun Yat-sen University（中山大学）；City University of Hong Kong（香港城市大学）；Xiangtan University（湘潭大学）
Automated negotiation is a rising topic in Artificial Intelligence research. Monte Carlo methods have got increasing interest, in particular since they have been used with success on games with high branching factor such as go.In this paper, we describe an Monte Carlo Negotiating Agent (MoCaNA) whose bidding strategy relies on Monte Carlo Tree Search. We provide our agent with opponent modeling tehcniques for bidding strtaegy and utility. MoCaNA can negotiate on continuous negotiating domains and in a context where no bound has been specified. We confront MoCaNA and the finalists of ANAC 2014 …
