Online auctions are one of the most fundamental facets of the modern economy and power an industry generating hundreds of billions of dollars a year in revenue. Auction theory has historically focused on the question of designing the best way to sell a single item to potential buyers, with the concurrent objectives of maximizing revenue generated or welfare created. Theoretical results in this area have typically relied on some prior Bayesian knowledge agents were assumed to have on each-other. This assumption is no longer satisfied in new markets such as online advertising: similar items are sold repeatedly, and agents are unaware of each other or might try to manipulate each-other. On the other hand, statistical learning theory now provides tools to supplement those missing pieces of information given enough data, as agents can learn from their environment to improve their strategies. This survey covers recent advances in learning in repeated auctions, starting from the traditional economic study of optimal one-shot auctions with a Bayesian prior. We then focus on the question of learning optimal mechanisms from a dataset of bidders' past values. The sample complexity as well as the computational efficiency of different methods will be studied. We will also investigate online variants where gathering data has a cost to be accounted for, either by seller or buyers ("earning while learning"). Later in the survey, we will further assume that bidders are also adaptive to the mechanism as they interact repeatedly with the same seller. We will show how strategic agents can actually manipulate repeated auctions, to their own advantage. All the questions discussed in this survey are grounded in real-world applications and many of the ideas and algorithms we describe are used every day to power the Internet economy.
We study revenue optimization in a repeated auction between a single seller and a single buyer. Traditionally, the design of repeated auctions requires strong modeling assumptions about the bidder behavior, such as it being myopic, infinite lookahead, or some specific form of learning behavior. Is it possible to design mechanisms which are simultaneously optimal against a multitude of possible buyer behaviors? We answer this question by designing a simple state-based mechanism that is simultaneously approximately optimal against a $k$-lookahead buyer for all $k$, a buyer who is a no-regret lea…
We study a general class of repeated auctions, such as the ones found in electricity markets, as multi-agent games between the bidders. In such a repeated setting, bidders can adapt their strategies online based on the data observed in the previous auction rounds. Moreover, if no-regret algorithms are employed by the bidders to update their strategies, the game is known to converge to a coarse-correlated equilibrium, which generalizes the notion of Nash equilibrium to a probabilistic view of the auction state. Well-studied no-regret algorithms depend on the feedback information available at ev…
With the increasing use of auctions in online advertising, there has been a large effort to study seller revenue maximization, following Myerson's seminal work, both theoretically and practically. We take the point of view of the buyer in classical auctions and ask the question of whether she has an incentive to shade her bid even in auctions that are reputed to be truthful, when aware of the revenue optimization mechanism. We show that in auctions such as the Myerson auction or a VCG with reserve price set as the monopoly price, the buyer who is aware of this information has indeed an incenti…
We consider the practical and classical setting where the seller is using an exploration stage to learn the value distributions of the bidders before running a revenue-maximizing auction in a exploitation phase. In this two-stage process, we exhibit practical, simple and robust strategies with large utility uplifts for the bidders. We quantify precisely the seller revenue against non-discounted buyers, complementing recent studies that had focused on impatient/heavily discounted buyers. We also prove the robustness of these shading strategies to sample approximation error of the seller, to bid…
The connection between games and no-regret algorithms has been widely studied in the literature. A fundamental result is that when all players play no-regret strategies, this produces a sequence of actions whose time-average is a coarse-correlated equilibrium of the game. However, much less is known about equilibrium selection in the case that multiple equilibria exist. In this work, we study the convergence of no-regret bidding algorithms in auctions. Besides being of theoretical interest, bidding dynamics in auctions is an important question from a practical viewpoint as well. We study repea…
Motivated by online display ad exchanges, we study a setting in which an exchange repeatedly interacts with bidders who have quota, making decisions about which subsets of bidders are called to participate in ad-slot-specific auctions. A bidder with quota cannot respond to more than a certain number of calls per second. In practice, random throttling is the principal solution by which these constraints are enforced. Given the repeated nature of the interaction with its bidders, the exchange has access to data containing information about each bidder's segments of interest. This information can…
As mobile devices have been ubiquitous, participatory sensing emerges as a powerful tool to solve many contemporary real life problems. Here, we contemplate the participatory sensing in online double auction environment by considering the location information of the participating agents. In this paper, we propose a truthful mechanism in this setting and the mechanism also satisfies the other economic properties such as budget balance and individual rationality.
Agents (specially humans) with smart devices are stemming with astounding rapidity and that may play a big role in information and communication technology apart from being used only as a mere calling devices. Inculcating the power of smart devices carried by the agents in several different applications is commonly termed as participatory sensing (PS). In this paper, for the first time a truthful quality adaptive participatory sensing is presented in an online double auction environment. The proposed algorithm is simulated with a benchmark mechanism that adapts the existing McAfee's Double Auc…
We study a basic auction design problem with online supply. There are two unit-demand bidders and two types of items. The first item type will arrive first for sure, and the second item type may or may not arrive. The auctioneer has to decide the allocation of an item immediately after each item arrives, but is allowed to compute payments after knowing how many items arrived. For this problem we show that there is no deterministic truthful and individually rational mechanism that, even with unbounded computational resources, gets any finite approximation factor to the optimal social welfare.
Interactions between bids to show ads online can lead to an advertiser's ad being shown to more men than women even when the advertiser does not target towards men. We design bidding strategies that advertisers can use to avoid such emergent discrimination without having to modify the auction mechanism. We mathematically analyze the strategies to determine the additional cost to the advertiser for avoiding discrimination, proving our strategies to be optimal in some settings. We use simulations to understand other settings.
For the scalability of industrial online advertising systems, a two-stage auction architecture is widely used to enable efficient ad allocation on a large set of corpus within a limited response time. The current deployed two-stage ad auction usually retrieves an ad subset by a coarse ad quality metric in a pre-auction stage, and then determines the auction outcome by a refined metric in the subsequent stage. However, this simple and greedy solution suffers from serious performance degradation, as it regards the decision in each stage separately, leading to an improper ad selection metric for …
Recently the online advertising market has exhibited a gradual shift from second-price auctions to first-price auctions. Although there has been a line of works concerning online bidding strategies in first-price auctions, it still remains open how to handle budget constraints in the problem. In the present paper, we initiate the study for a buyer with budgets to learn online bidding strategies in repeated first-price auctions. We propose an RL-based bidding algorithm against the optimal non-anticipating strategy under stationary competition. Our algorithm obtains $\widetilde O(\sqrt T)$-regre…
We study revenue optimization pricing algorithms for repeated posted-price auctions where a seller interacts with a single strategic buyer that holds a fixed private valuation. We show that, in the case when both the seller and the buyer have the same discounting in their cumulative utilities (revenue and surplus), there exist two optimal algorithms. The first one constantly offers the Myerson price, while the second pricing proposes a "big deal": pay for all goods in advance (at the first round) or get nothing. However, when there is an imbalance between the seller and the buyer in the patien…
A large fraction of online advertisement is sold via repeated second price auctions. In these auctions, the reserve price is the main tool for the auctioneer to boost revenues. In this work, we investigate the following question: Can changing the reserve prices based on the previous bids improve the revenue of the auction, taking into account the long-term incentives and strategic behavior of the bidders? We show that if the distribution of the valuations is known and satisfies the standard regularity assumptions, then the optimal mechanism has a constant reserve. However, when there is uncert…
University of Luxembourg（卢森堡大学）；Télécom Paris（巴黎电信学院）；Université de Toulouse - IRIT（图卢兹大学 - 图卢兹信息研究所）；University of Naples Federico II（那不勒斯费德里科二世大学）
In online advertising, search engines sell ad placements for keywords continuously through auctions. This problem can be seen as an infinitely repeated game since the auction is executed whenever a user performs a query with the keyword. As advertisers may frequently change their bids, the game will have a large set of equilibria with potentially complex strategies. In this paper, we propose the use of natural strategies for reasoning in such setting as they are processable by artificial agents with limited memory and/or computational power as well as understandable by human users. To reach th…
We consider an online ad network problem in which an ad exchange auctions ad slots and intermediaries called demand side platforms (DSPs) buy these ad slots for their clients (advertisers). An intermediary represents multiple advertisers. Different types of ad slots are auctioned by the ad exchange, e.g., video ad, banner ad etc. We study repeated posted price auctions for homogeneous and heterogeneous items when there is an intermediary. In a posted price auction, the auctioneer sets a fixed reserve price. The buyer can accept the price and win the ad slot or reject the price. We analyze the …
Small operators who take part in secondary wireless spectrum markets typically have strict budget limits. In this paper, we study the bidding problem of a budget constrained operator in repeated secondary spectrum auctions. In existing truthful auctions, truthful bidding is the optimal strategy of a bidder. However, budget limits impact bidding behaviors and make bidding decisions complicated, since bidders may behave differently to avoid running out of money. We formulate the problem as a dynamic auction game between operators, where knowledge of other operators is limited due to the distribu…
We propose a multi-agent distributed reinforcement learning algorithm that balances between potentially conflicting short-term reward and sparse, delayed long-term reward, and learns with partial information in a dynamic environment. We compare different long-term rewards to incentivize the algorithm to maximize individual payoff and overall social welfare. We test the algorithm in two simulated auction games, and demonstrate that 1) our algorithm outperforms two benchmark algorithms in a direct competition, with cost to social welfare, and 2) our algorithm's aggressive competitive behavior ca…
We study revenue optimization learning algorithms for repeated second-price auctions with reserve where a seller interacts with multiple strategic bidders each of which holds a fixed private valuation for a good and seeks to maximize his expected future cumulative discounted surplus. We propose a novel algorithm that has strategic regret upper bound of $O(\log\log T)$ for worst-case valuations. This pricing is based on our novel transformation that upgrades an algorithm designed for the setup with a single buyer to the multi-buyer case. We provide theoretical guarantees on the ability of a tra…
We study the online learning problem of a bidder who participates in repeated auctions. With the goal of maximizing his T-period payoff, the bidder determines the optimal allocation of his budget among his bids for $K$ goods at each period. As a bidding strategy, we propose a polynomial-time algorithm, inspired by the dynamic programming approach to the knapsack problem. The proposed algorithm, referred to as dynamic programming on discrete set (DPDS), achieves a regret order of $O(\sqrt{T\log{T}})$. By showing that the regret is lower bounded by $Ω(\sqrt{T})$ for any strategy, we conclude tha…
Online auction has been very widespread in the recent years. Platform administrators are working hard to refine their auction mechanisms that will generate high profits while maintaining a fair resource allocation. With the advancement of computing technology and the bottleneck in theoretical frameworks, researchers are shifting gears towards online auction designs using deep learning approaches. In this article, we summarized some common deep learning infrastructures adopted in auction mechanism designs and showed how these architectures are evolving. We also discussed how researchers are tac…
The Hebrew University of Jerusalem（耶路撒冷希伯来大学）；Microsoft Research（微软研究院）
We consider the problem of bid prediction in repeated auctions and evaluate the performance of econometric methods for learning agents using a dataset from a mainstream sponsored search auction marketplace. Sponsored search auctions is a billion dollar industry and the main source of revenue of several tech giants. A critical problem in optimizing such marketplaces is understanding how bidders will react to changes in the auction design. We propose the use of no-regret based econometrics for bid prediction, modeling players as no-regret learners with respect to a utility function, unknown to t…
In this work, we investigate online mechanisms for trading time-sensitive valued data. We adopt a continuous function $d(t)$ to represent the data value fluctuation over time $t$. Our objective is to design an \emph{online} mechanism achieving \emph{truthfulness} and \emph{revenue-competitiveness}. We first prove several lower bounds on the revenue competitive ratios under various assumptions. We then propose several online truthful auction mechanisms for various adversarial models, such as a randomized observe-then-select mechanism $\mathcal{M}_1$ and prove that it is \textit{truthful} and $Θ…
Large Language Models as Bidding Agents in Repeated HetNet Auction
College of Science and Engineering（科学与工程学院）；Hamad Bin Khalifa University（哈马德·本·卡西姆大学）；CNRS（国家科学研究中心）；Universite de Lorraine（洛林大学）；CRAN（CRAN研究中心）；Department of Computer Science（计算机科学系）；Khalifa University（卡西姆大学）
On ad exchange platforms the place for advertisement is sold through different kinds of auctions. However, it is not uncommon the situation where the seller repeatedly encounters only one buyer, thus the posted price auction degenerates into a monopoly-monopsony game with asymmetric information and nearly an infinite number of rounds; on each round the seller proposes the price and the buyer accepts or rejects it. I learned this problem from a discussion with members of Yandex research team and my main motivation was to find an incentive-compatible seller's strategy. In this short paper such a…
Ad exchanges are kind of the most popular online advertising marketplaces for trading ad spaces over the Internet. Ad exchanges run auctions to sell diverse ad spaces on the publishers' web-pages to advertisers, who want to display ads on ad spaces. However, the parties in an ad auction cannot verify whether the auction is carried out correctly or not. Furthermore, the advertisers are usually unwilling to reveal their sensitive bids and identities. In this paper, we jointly consider the auction verifiability and the advertisers' privacy preservation, and thus propose ERA, which is an Efficient…
With the advent and increasing consolidation of e-commerce, digital advertising has very recently replaced traditional advertising as the main marketing force in the economy. In the past four years, a particularly important development in the digital advertising industry is the shift from second-price auctions to first-price auctions for online display ads. This shift immediately motivated the intellectually challenging question of how to bid in first-price auctions, because unlike in second-price auctions, bidding one's private value truthfully is no longer optimal. Following a series of rece…
In display advertising, a small group of sellers and bidders face each other in up to 10 12 auctions a day. In this context, revenue maximisation via monopoly price learning is a high-value problem for sellers. By nature, these auctions are online and produce a very high frequency stream of data. This results in a computational strain that requires algorithms be real-time. Unfortunately, existing methods inherited from the batch setting suffer O($\sqrt t$) time/memory complexity at each update, prohibiting their use. In this paper, we provide the first algorithm for online learning of monopoly…
We propose an auction for online advertising where each ad occupies either one square or two horizontally-adjacent squares of a grid of squares. Our primary application are ads for products shown on retail websites such as Instacart or Amazon where the products are naturally organized into a grid. We propose efficient algorithms for computing the optimal layout of the ads and pricing of the ads. The auction is a generalization of the generalized second-price (GSP) auction used by internet search engines (e.g. Google, Microsoft Bing, Yahoo!).
