Game theory has been developed by scientists as a theory of strategic interaction among players who are supposed to be perfectly rational. These strategic interactions might have been presented in an auction, a business negotiation, a chess game, or even in a political conflict aroused between different agents. In this study, the strategic (rational) agents created by reinforcement learning algorithms are supposed to be bidder agents in various types of auction mechanisms such as British Auction, Sealed Bid Auction, and Vickrey Auction designs. Next, the equilibrium points determined by the agents are compared with the outcomes of the Nash equilibrium points for these environments. The bidding strategy of the agents is analyzed in terms of individual rationality, truthfulness (strategy-proof), and computational efficiency. The results show that using a multi-agent reinforcement learning strategy improves the outcomes of the auction simulations.
All-pay auctions, a common mechanism for various human and agent interactions, suffers, like many other mechanisms, from the possibility of players' failure to participate in the auction. We model such failures, and fully characterize equilibrium for this class of games, we present a symmetric equilibrium and show that under some conditions the equilibrium is unique. We reveal various properties of the equilibrium, such as the lack of influence of the most-likely-to-participate player on the behavior of the other players. We perform this analysis with two scenarios: the sum-profit model, where…
The market economy deals with many interacting agents such as buyers and sellers who are autonomous intelligent agents pursuing their own interests. One such multi-agent system (MAS) that plays an important role in auctions is the combinatorial auctioning system (CAS). We use this framework to define our concept of fairness in terms of what we call as "basic fairness" and "extended fairness". The assumptions of quasilinear preferences and dominant strategies are taken into consideration while explaining fairness. We give an algorithm to ensure fairness in a CAS using a Generalized Vickrey Auct…
We study simple and approximately optimal auctions for agents with a particular form of risk-averse preferences. We show that, for symmetric agents, the optimal revenue (given a prior distribution over the agent preferences) can be approximated by the first-price auction (which is prior independent), and, for asymmetric agents, the optimal revenue can be approximated by an auction with simple form. These results are based on two technical methods. The first is for upper-bounding the revenue from a risk-averse agent. The second gives a payment identity for mechanisms with pay-your-bid semantics…
In this paper we present and evaluate a general framework for the design of truthful auctions for matching agents in a dynamic, two-sided market. A single commodity, such as a resource or a task, is bought and sold by multiple buyers and sellers that arrive and depart over time. Our algorithm, Chain, provides the first framework that allows a truthful dynamic double auction (DA) to be constructed from a truthful, single-period (i.e. static) double-auction rule. The pricing and matching method of the Chain construction is unique amongst dynamic-auction rules that adopt the same building block. …
In this study, we apply reinforcement learning techniques and propose what we call reinforcement mechanism design to tackle the dynamic pricing problem in sponsored search auctions. In contrast to previous game-theoretical approaches that heavily rely on rationality and common knowledge among the bidders, we take a data-driven approach, and try to learn, over repeated interactions, the set of optimal reserve prices. We implement our approach within the current sponsored search framework of a major search engine: we first train a buyer behavior model, via a real bidding data set, that accuratel…
We consider prior-free auctions for revenue and welfare maximization when agents have a common budget. The abstract environments we consider are ones where there is a downward-closed and symmetric feasibility constraint on the probabilities of service of the agents. These environments include position auctions where slots with decreasing click-through rates are auctioned to advertisers. We generalize and characterize the envy-free benchmark from Hartline and Yan (2011) to settings with budgets and characterize the optimal envy-free outcomes for both welfare and revenue. We give prior-free mech…
Combinatorial auctions where agents can bid on bundles of items are desirable because they allow the agents to express complementarity and substitutability between the items. However, expressing one's preferences can require bidding on all bundles. Selective incremental preference elicitation by the auctioneer was recently proposed to address this problem (Conen & Sandholm 2001), but the idea was not evaluated. In this paper we show, experimentally and theoretically, that automated elicitation provides a drastic benefit. In all of the elicitation schemes under study, as the number of items for…
We analyze a scenario in which software agents implemented as regret-minimizing algorithms engage in a repeated auction on behalf of their users. We study first-price and second-price auctions, as well as their generalized versions (e.g., as those used for ad auctions). Using both theoretical analysis and simulations, we show that, surprisingly, in second-price auctions the players have incentives to misreport their true valuations to their own learning agents, while in the first-price auction it is a dominant strategy for all players to truthfully report their valuations to their agents.
We propose a multi-agent distributed reinforcement learning algorithm that balances between potentially conflicting short-term reward and sparse, delayed long-term reward, and learns with partial information in a dynamic environment. We compare different long-term rewards to incentivize the algorithm to maximize individual payoff and overall social welfare. We test the algorithm in two simulated auction games, and demonstrate that 1) our algorithm outperforms two benchmark algorithms in a direct competition, with cost to social welfare, and 2) our algorithm's aggressive competitive behavior ca…
We design approximate weakly group strategy-proof mechanisms for resource reallocation problems using Milgrom and Segal's deferred acceptance auction framework: the radio spectrum and network bandwidth reallocation problems in the procurement auction setting and the cost minimization problem with set cover constraints in the selling auction setting. Our deferred acceptance auctions are derived from simple greedy algorithms for the underlying optimization problems and guarantee approximately optimal social welfare (cost) of the agents retaining their rights (contracts). In the reallocation prob…
We study the problem of a buyer (aka auctioneer) who gains stochastic rewards by procuring multiple units of a service or item from a pool of heterogeneous strategic agents. The reward obtained for a single unit from an allocated agent depends on the inherent quality of the agent; the agent's quality is fixed but unknown. Each agent can only supply a limited number of units (capacity of the agent). The costs incurred per unit and capacities are private information of the agents. The auctioneer is required to elicit costs as well as capacities (making the mechanism design bidimensional) and fur…
This paper is aimed at the stipulations which arise in the traditional online auctions as a result of various anomalies in the reputation and trust calculation mechanism. We try to improve the scalability and efficiency of the online auctions by providing efficient trust management methodology considering several factors into consideration. A comparison between the performance of the auctions system with and without the agent methodology is done with good results
This paper investigates the problem of proportionally fair double sided energy auction involving buying and selling agents. The grid is assumed to be operating under islanded mode. A distributed auction algorithm that can be implemented by an aggregator, as well as a possible approach by which the agents may approximate price anticipation is considered. Equilibrium conditions arising due to price anticipation is analyzed. A modified auction to mitigate the resulting loss in efficiency due to such behavior is suggested. This modified auction allows the aggregate social welfare of the agents to …
We study the design of mechanisms in combinatorial auction domains. We focus on settings where the auction is repeated, motivated by auctions for licenses or advertising space. We consider models of agent behaviour in which they either apply common learning techniques to minimize the regret of their bidding strategies, or apply short-sighted best-response strategies. We ask: when can a black-box approximation algorithm for the base auction problem be converted into a mechanism that approximately preserves the original algorithm's approximation factor on average over many iterations? We present…
Empirical game-theoretic analysis (EGTA) has recently been applied successfully to analyze the behavior of large numbers of competing traders in a continuous double auction market. Multiagent simulation methods like EGTA are useful for studying complex strategic environments like a stock market, where it is not feasible to solve analytically for the rational behavior of each agent. A weakness of simulation-based methods in strategic settings, however, is that it is typically impossible to prove that the strategy profile assigned to the simulated agents is stable, as in a Nash equilibrium. I pr…
Continuous double auctions such as the limit order book employed by exchanges are widely used in practice to match buyers and sellers of a variety of financial instruments. In this work, we develop an agent-based model for trading in a limit order book and show (1) how opponent modelling techniques can be applied to classify trading agent archetypes and (2) how behavioural cloning can be used to imitate these agents in a simulated setting. We experimentally compare a number of techniques for both tasks and evaluate their applicability and use in real-world scenarios.
The idea of this paper is an advanced game concept. This concept is expected to model non-monetary bilateral cooperations between self-interested agents. Such non-monetary cases are social cooperations like allocation of high level jobs or sexual relationships among humans. In a barter double auction, there is a big amount of agents. Every agent has a vector of parameters which specifies his demand and a vector which specifies his offer. Two agents can achieve a commitment through barter exchange. The subjective satisfaction level (a number between 0% and 100%) of an agent is as high as small …
Distributed energy resources (DERs), such as rooftop solar panels, are growing rapidly and are reshaping power systems. To promote DERs, feed-in-tariff (FIT) is usually adopted by utilities to pay DER owners certain fixed rates for supplying energy to the grid. An alternative to FIT is a market-based approach; that is, consumers and DER owners trade energy in an auction-based peer-to-peer (P2P) market, and the rates are determined based on supply and demand. However, the auction complexity and market participants' bounded rationality may invalidate many well-established theories on auction des…
The revenue optimal mechanism for selling a single item to agents with independent but non-identically distributed values is complex for agents with linear utility (Myerson,1981) and has no closed-form characterization for agents with non-linear utility (cf. Alaei et al., 2012). Nonetheless, for linear utility agents satisfying a natural regularity property, Alaei et al. (2018) showed that simply posting an anonymous price is an e-approximation. We give a parameterization of the regularity property that extends to agents with non-linear utility and show that the approximation bound of anonymou…
We study truthful mechanisms for hiring a team of agents in three classes of set systems: Vertex Cover auctions, k-flow auctions, and cut auctions. For Vertex Cover auctions, the vertices are owned by selfish and rational agents, and the auctioneer wants to purchase a vertex cover from them. For k-flow auctions, the edges are owned by the agents, and the auctioneer wants to purchase k edge-disjoint s-t paths, for given s and t. In the same setting, for cut auctions, the auctioneer wants to purchase an s-t cut. Only the agents know their costs, and the auctioneer needs to select a feasible set …
This paper considers prior-independent mechanism design, namely identifying a single mechanism that has near optimal performance on every prior distribution. We show that mechanisms with truthtelling equilibria, a.k.a., revelation mechanisms, do not always give optimal prior-independent mechanisms and we define the revelation gap to quantify the non-optimality of revelation mechanisms. This study suggests that it is important to develop a theory for the design of non-revelation mechanisms. Our analysis focuses on welfare maximization in single-item auctions for agents with budgets and a natura…
We study the efficiency guarantees in the simple auction environment where the auctioneer has one unit of divisible good to be distributed among a number of budget constrained agents. With budget constraints, the social welfare cannot be approximated by a better factor than the number of agents by any truthful mechanism. Thus, we follow a recent work by Dobzinski and Leme (ICALP 2014) to approximate the liquid welfare, which is the welfare of the agents each capped by her/his own budget. We design a new truthful auction with an approximation ratio of $\frac{\sqrt{5}+1}{2} \approx 1.618$, impro…
We consider some classical optimization problems in path planning and network transport, and we introduce new auction-based algorithms for their optimal and suboptimal solution. The algorithms are based on mathematical ideas that are related to competitive bidding by persons for objects and the attendant market equilibrium, which underlie auction processes. However, the starting point of our algorithms is different, namely weighted and unweighted path construction in directed graphs, rather than assignment of persons to objects. The new algorithms have several potential advantages over existin…
The planner wants to give k identical, indivisible objects to the top k valuation agents at zero costs. Each agent knows her own valuation of the object and whether it is among the top k. Modify the (k+1)st-price sealed-bid auction by introducing a small participation fee and the option not to participate in it. This simple mechanism implements the desired outcome in iteratively undominated strategies. Moreover, no pair of agents can profitably deviate from the equilibrium by coordinating their strategies or bribing each other.
This research proposes a novel auction mechanism for transactive energy exchange between buyers and sellers, modeled as agents in a microgrid. The mechanism is implemented by a separate microgrid controller (MC) agent, and requires big data flow with the other agents through an iterative bidding process. Although private user information remains hidden to the MC, a theoretical analysis shows that under the assumption of convexity of the agents utilities, the proposed auction is still able to maximize the social welfare (SW), i.e. the aggregate utilities of the agents. In addition, it is shown …
We study an abstract optimal auction problem for a single good or service. This problem includes environments where agents have budgets, risk preferences, or multi-dimensional preferences over several possible configurations of the good (furthermore, it allows an agent's budget and risk preference to be known only privately to the agent). These are the main challenge areas for auction theory. A single-agent problem is to optimize a given objective subject to a constraint on the maximum probability with which each type is allocated, a.k.a., an allocation rule. Our approach is a reduction from m…
Constraints on agent's ability to pay play a major role in auction design for any setting where the magnitude of financial transactions is sufficiently large. Those constraints have been traditionally modeled in mechanism design as \emph{hard budget}, i.e., mechanism is not allowed to charge agents more than a certain amount. Yet, real auction systems (such as Google AdWords) allow more sophisticated constraints on agents' ability to pay, such as \emph{average budgets}. In this work, we investigate the design of Pareto optimal and incentive compatible auctions for agents with \emph{constrained…
We study the efficiency of simple combinatorial auctions for the allocation of a set of items to a set of agents, with private subadditive valuation functions and budget constraints. The class we consider includes all auctions that allocate each item independently to the agent that submits the highest bid for it, and requests a payment that depends on the bids of all agents only for this item. Two well-known examples of this class are the simultaneous first and second price auctions. We focus on the pure equilibria of the induced strategic games, and using the liquid welfare as our efficiency …
University of Zurich（苏黎世大学）；Google Research（谷歌研究院）
We cast the problem of combinatorial auction design in a Bayesian framework in order to incorporate prior information into the auction process and minimize the number of rounds to convergence. We first develop a generative model of agent valuations and market prices such that clearing prices become maximum a posteriori estimates given observed agent valuations. This generative model then forms the basis of an auction process which alternates between refining estimates of agent valuations and computing candidate clearing prices. We provide an implementation of the auction using assumed density …
In this paper, we study sequential auctions with two budget constrained bidders and any number of identical items. All prior results on such auctions consider only two items. We construct a canonical outcome of the auction that is the only natural equilibrium and is unique under a refinement of subgame perfect equilibria. We show certain interesting properties of this equilibrium; for instance, we show that the prices decrease as the auction progresses. This phenomenon has been observed in many experiments and previous theoretic work attributed it to features such as uncertainty in the supply …
We consider budget feasible mechanisms for procurement auctions with additive valuation functions. For the divisible case, where agents can be allocated fractionally, there exists an optimal mechanism with approximation guarantee $e/(e-1)$ under the small bidder assumption. We study the divisible case without the small bidder assumption, but assume that the true costs of the agents are bounded by the budget. This setting lends itself to modeling economic situations in which the goods represent time and the agents' true costs are not necessarily small compared to the budget. Non-trivially, we g…
This paper studies a mechanism design problem over a network, where agents can only participate by referrals. The Bulow-Klemberer theorem proposes that expanding the number of participants is a more effective approach to increase revenue than modifying the auction format. However, agents lack the motivation to invite others because doing so intensifies competition among them. On the other hand, misreporting social networks is also a common problem that can reduce revenue. Examples of misreporting include Sybil attacks (an agent pretending to be multiple bidders) and coalition groups (multiple …
We present a new method for multi-agent planning involving human drivers and autonomous vehicles (AVs) in unsignaled intersections, roundabouts, and during merging. In multi-agent planning, the main challenge is to predict the actions of other agents, especially human drivers, as their intentions are hidden from other agents. Our algorithm uses game theory to develop a new auction, called GamePlan, that directly determines the optimal action for each agent based on their driving style (which is observable via commonly available sensors). GamePlan assigns a higher priority to more aggressive or…
Imperial College London（帝国理工学院）；Aston University（阿斯顿大学）
We consider an outsourcing problem where a software agent procures multiple services from providers with uncertain reliabilities to complete a computational task before a strict deadline. The service consumer requires a procurement strategy that achieves the optimal balance between success probability and invocation cost. However, the service providers are self-interested and may misrepresent their private cost information if it benefits them. For such settings, we design a novel procurement auction that provides the consumer with the highest possible revenue, while giving sufficient incentive…
We present LADDER, the first deep reinforcement learning agent that can successfully learn control policies for large-scale real-world problems directly from raw inputs composed of high-level semantic information. The agent is based on an asynchronous stochastic variant of DQN (Deep Q Network) named DASQN. The inputs of the agent are plain-text descriptions of states of a game of incomplete information, i.e. real-time large scale online auctions, and the rewards are auction profits of very large scale. We apply the agent to an essential portion of JD's online RTB (real-time bidding) advertisin…
Agents (specially humans) with smart devices are stemming with astounding rapidity and that may play a big role in information and communication technology apart from being used only as a mere calling devices. Inculcating the power of smart devices carried by the agents in several different applications is commonly termed as participatory sensing (PS). In this paper, for the first time a truthful quality adaptive participatory sensing is presented in an online double auction environment. The proposed algorithm is simulated with a benchmark mechanism that adapts the existing McAfee's Double Auc…
This paper proves that the welfare of the first price auction in Bayes-Nash equilibrium is at least a $.743$-fraction of the welfare of the optimal mechanism assuming agents' values are independently distributed. The previous best bound was $1-1/e \approx .63$, derived in Syrgkanis and Tardos (2013) using smoothness, the standard technique for reasoning about welfare of games in equilibrium. In the worst known example (from Hartline et al. (2014)), the first price auction achieves a $\approx .869$-fraction of the optimal welfare, far better than the theoretical guarantee. Despite this large ga…
This paper considers an electric vehicle charging scheduling setting where vehicle users can reserve charging time in advance at a charging station. In this setting, users are allowed to explicitly express their preferences over different start times and the length of charging periods for charging their vehicles. The goal is to compute optimal charging schedules which maximize the social welfare of all users given their time preferences and the state of charge of their vehicles. Assuming that users are self-interested agents who may behave strategically to advance their own benefits rather tha…
The intuition that profit is optimized by maximizing marginal revenue is a guiding principle in microeconomics. In the classical auction theory for agents with linear utility and single-dimensional preferences, Bulow and Roberts (1989) show that the optimal auction of Myerson (1981) is in fact optimizing marginal revenue. In particular Myerson's virtual values are exactly the derivative of an appropriate revenue curve. This paper considers mechanism design in environments where the agents have multi-dimensional and non-linear preferences. Understanding good auctions for these environments is c…
In a seminal paper, McAfee (1992) presented a truthful mechanism for double auctions, attaining asymptotically-optimal gain-from-trade without any prior information on the valuations of the traders. McAfee's mechanism handles single-parametric agents, allowing each seller to sell a single unit and each buyer to buy a single unit. This paper presents a double-auction mechanism that handles multi-parametric agents and allows multiple units per trader, as long as the valuation functions of all traders have decreasing marginal returns. The mechanism is prior-free, ex-post individually-rational, do…
Simultaneous ascending auctions present agents with the exposure problem: bidding to acquire a bundle risks the possibility of obtaining an undesired subset of the goods. Auction theory provides little guidance for dealing with this problem. We present a new family of decisiontheoretic bidding strategies that use probabilistic predictions of final prices. We focus on selfconfirming price distribution predictions, which by definition turn out to be correct when all agents bid decision-theoretically based on them. Bidding based on these is provably not optimal in general, but our experimental ev…
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；University of Waterloo（滑铁卢大学）
Recent work has suggested reducing electricity generation cost by cutting the peak to average ratio (PAR) without reducing the total amount of the loads. However, most of these proposals rely on consumer's willingness to act. In this paper, we propose an approach to cut PAR explicitly from the supply side. The resulting cut loads are then distributed among consumers by the means of a multiunit auction which is done by an intelligent agent on behalf of the consumer. This approach is also in line with the future vision of the smart grid to have the demand side matched with the supply side. Exper…
We consider the question of whether collusion among bidders (a "bidding ring") can be supported in equilibrium of unrepeated first-price auctions. Unlike previous work on the topic such as that by McAfee and McMillan [1992] and Marshall and Marx [2007], we do not assume that non-colluding agents have perfect knowledge about the number of colluding agents whose bids are suppressed by the bidding ring, and indeed even allow for the existence of multiple cartels. Furthermore, while we treat the association of bidders with bidding rings as exogenous, we allow bidders to make strategic decisions ab…
