Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning

Game Theoretic Understanding of Social Economic System Design

Xiaotie Deng

Department of Computer Science Shanghai Jiaotong University

November 22, 2015

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Main Content

Game Theory in the Era of the Internet

An Example of P2P Bandwidth Sharing

Learn Auction ?

Learning and Nash Equilibrium

The Game Theoretical Challenges in Machine Learning

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning

Game Theory in the Era of the Internet

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning The Era of the Internet

• Email: never made it to profitability • Hypertext documents over WWW: Wiki, Search Engine Economics • Society online: facebook(mobile ads), WeChat(ads in disguise) • P2P networks: BitTorrent, Open Garden • Internet finance: BitCoin, Alipay’s Yu’ebao and three thousands more new companies...

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Force of Stability

• Fairness and Unity: Each is treated fairly • A Example of Core: no subset can be better off on their own • Truth Telling in Mechanism Design: truthful player always achieves maximum utility • Equilibrium: No player can gain an advantage by changing its strategy • Truful bid equilibrium

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Bounded Rationality: as quantified by computer science

The extra effort for optimality may not be worthy of the effort • Time Complexity: P vs NP • Price of Anarchy: Benefit of centrolized protocols • Incentive Ratio: every agent has limited incentive to cheat • Approximate Nash Equilibrium: every agent’s strategy is approximately optimal

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Applications with Multiple Agents

• Cooperation and Competition: Platforms of Mobile Services • Sharing Economy: Uber • Adversarial Participants: Credit Card Fraud • Asymmetry of Information: Goods of (Unknown, Probability) Value

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning

An Example of P2P Bandwidth Sharing

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning An Example of P2P Bandwidth Sharing

Undirected Graph G = (V , E; w) • V : nodes of the network. Each is owned by an agent. • E: communication channels between two agents. • w : V → N: w(u) the bandwidth, (e.g., the number of movies uploaded by u) of u which can be transferred to its neighbours. • utility of a user is the number of movies received from its neighbours (fraction amount is fine). Required output: Each node sent all its movies to its neighbours but one movie is sent to one neighbour only.

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Fairness: Proportional Response

• Provide each a share of in proportion to what YOU receives from others.

• Let aji , j ∈ Γi , be what node i receives from its neighbors. • Let wi be the total amount node i is to give out to others. ∑ aji • Node will give agent j, a total bandwidth wij = wi ∗ ∈ ati t Γi

• FIXED point: ∀(i, j)aij = wij , i.e., update returns the same value.

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Market Equilibrium

• Each has an item: Node u has a bandwidth weight wu. • The bandwidth of each node will have a (different) price: the price of wu is pu. • Every node agent wants to buy in total as much bandwidth as possible from its neighbors. • Market equilibrium (price) • Everyone selects its purchase that maximize its utility • Market clearance: Bandwidth of agent u is sold out or pu = 0.

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Euivalence of Fairness and Competitiveness

(Wu and Zhang, STOC’2007) • Proportional response has a fixed point which is computable in polynomial time. • Start with evenly distribution of everyone’s bandwidth to its neighbours, proportional response converges to the fixed point. • The fixed point of proportional response is a market equilibrium, • They can be found in polynomial time.

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Incentive Challenge

• Could one cheat to increase the total bandwidth it eventually receives ? • (Cheng, Deng, Pi, Yan, SAGT 2015) • No player can improve its utility by cutting its own communication channels

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Summary: a network protocol relating solution concepts

• Fixed point of a fair protocol • Market equilibrium • Truthful play Nash equilibrium

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning

Learn Auction ?

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Information Asymmetry at the Market Place

• Market maker knows all the prices (and bids) • Each market participant knows own private value • Goal of auction design: discovery of prices from unknown private values

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Hypothetical Approach by a Machine Learner

• Machine Learner tries to predict future prices from past experience • Each market participant knows own (private) value • Machine learner’s plan: use predicted prices to help charging the market participants.

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Tasks of two paradigms

• Economics: Find price and allocation to maximize social welfare, decided by demand, supply and utility functions. • Contribution of Learning: to understand the users. To predict true values? To predict bids or prices?

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Sponsored Search Auction

• Truth telling auctions 1. Vickrey Auction: the highest bidder wins and pays the second highest price. 2. Myerson Optimum Auction: reserved price auction. 3. Truthful mechanism: price discovery becomes value discovery. • Legacy Mechanism in Sponsored Search (Generalized Second Price) • Non-truthful • Forward looking Nash equilibrium recovers true values

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Sponsored Search Auction

• Truth telling auctions 1. Vickrey Auction: the highest bidder wins and pays the second highest price. 2. Myerson Optimum Auction: reserved price auction. 3. Truthful mechanism: price discovery becomes value discovery. • Legacy Mechanism in Sponsored Search GSP (Generalized Second Price) • Non-truthful

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning GSP

• Input Parameters • Multiple item market: n buyers and k items • Each buyer i has a value parameter αi known to itself • Each item j has a quality parameter βj publicly known, in decreasing order βj ≥ βj+1 • Values of an item j to buyer i is αi ∗ βj . • Auction Protocol

• Each buyer submit a bid bi (= αi if truthful) • Sort bi s, bi ≥ bi+1. • Allocate item i to buyer i. • Charge buyer i the price bi+1 ∗ βi . • Property • A buyer may cheat

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning GSP is not truthful

• Cheating example

• α1 = 5, α2 = 4, α3 = 1 • β1 = 5, β2 = 4 • Truthful bid solution • Buyer 1 gets item 1 pays 4 ∗ 5 = 20 with utility 25 − 20 = 5. • Buyer 2 gets item 2 pays 1 ∗ 4 = 4 with utility 4 ∗ 4 − 4 = 12. • Buyer 1 cheats • Buyer 1 bids 4.5. • Buyer 1 gets item 2 pays 1 ∗ 4 = 4 with utility 4 ∗ 5 − 4 = 16. • Buyer 1 increases its utility by 11.

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Optimal Bidding in GSP

• Forward looking bid • utility of a bidder determined by its bid where other bids are fixed • once decided to win item i, bid bi placed to disallow bidder of i − 1 to lower cut it. (αi − bi+1) ∗ βi = (αi − bi ) ∗ βi−1 ′ βi • the bid: b = αi − (αi − bi+1) i βi−1 • Forward looking Nash∑ equilibrium price m • ∗ ′ ′ − ′ pj = bj+1 βj = j′=j αj +1(βj βj +1), where βm+1 = 0.

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Truthful Bid Mechanism Design Dilemma

• Selling one item at each time period. • Day 1: Apply Vickrey Auction • Day 2: Apply reserved price auction: set the reserved price as the highest bid of Day 1

• Non-truthful GSP: recover αi s from Forward looking Nash equilibrium. • Related Approach in AI: A game-theoretic machine learning approach for revenue maximization in sponsored search (based on a Markovian model, by He,Chen,Wang,Liu, AAAI 2013).

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Machine Learning of Pricing

1. Learn Truth Value ? 2. Non-truthful mechanism (Generalized Second Price) • Forward looking Nash equilibrium: true values can be recovered. 3. Optimum auction in Bayesian setting • Learning of distribution from bids (not from true value).

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning

Learning and Nash Equilibrium

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Anonymous game

• n players [n] = {1, 2, ··· , n}, n ≥ 2. • S = {1, 2, ··· , s} strategies s ≥ 2. • There are a constant types of players each, T , each type has a constant number of strategies, |S|. • The payoff of a player depends on the number of players in a type playing a strategy over all the possible types and all the strategies.

• payoffi ((t, s)i , multiset{(t, s)j : j ∈ P−i }) • Input size: constant for each player where |S| and |T | are constant. Linear in the number of players.

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning A Mixed Nash Equilibrium of the Anonymous Game where each has two strategies

• Player i has a probability pi to play strategy 1, and the rest, playing strategy 0. Denote such a strategy as p⃗ =< p1, p2, ··· > • { ··· − } Notations: Let Xi be a random variable on 0∑, 1, , k 1 n for each i. Let Xi ’s be independent and X = i=1 Xi . ′ ′ • X is a Nash equilibrium if ui (X ) ≥ ui (X ) for any X with ∀ ̸ ′ j = i : Xj = Xj .

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Finding an approximate solution in Anonymous Games (started with Daskalakis and Papadimitriou 2008)

• There is a pure strategy ϵ-Nash equilibrium. • (Discretization) Given a mixed strategy Nash equilibrium p⃗, there exists a way to round it to a vector q⃗ such that

• q⃗ has all its component qi an integer multiple of ϵ • p⃗ and q⃗ are of a small total variance. • It implies the utility of each player will not change much. • 1 Code it into a game of k + 1 strategies, k = ϵ2 . Solve it as a pure Nash equilibrium.

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Recent Generalizations

• Discrete Learning Learn of unknown PMD (Poisson multinomial distribution) • Construct an ϵ-cover of the space of PMDs • Compute a set of ϵ-best response for each element. • Check if this element is in the set of best responses (loop back).

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Nash Equilibrium in Anonymous Games

• Xi Chen, David Durfee, and Anthi Orfanou • Find an ϵ-approaximate Nash Equilibrium in anonymous game is PPAD-complete • Each player has no more than 7 strategies

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Learning Sums of IID

Ilias Diakonikolas, Daniel Kane, and Alistair Stewart (2015)

• X = X1 + X2 + ··· + Xn where Xi ’s are mutually independently random variables supported on {0, 1, ··· , k − 1} • k = 2: Binomial distributions (BD) • k = 2: Poisson binomial distribution (PBD) • A polynomial time algorithm to learn it using θO(k/ϵ2) independent sample and takes O˜(k3/ϵ2) time. (Total variance ≤ ϵ with probability ≥ 2/3)

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Distribution-Free Testing of Monotone Conjunction

Xi Chen and Jinye Xie • Oracle of an unknown Boolean Function: f : {0, 1}n → {0, 1}. • Sample Oracle to unknown distribution D over {0, 1}. • O˜(n3/ϵ5)-query algorithm and Ω(˜ n3) lower bound • test whether f is monotone conjunctive

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Central Limit Theory on Poisson Multinomial Distributions (PMD)

Costis Daskalakis, Anindya De, Gautam Kamath and Christos Tzamos • Distribution for the sum of n independent random vectors supported on the Bk = {e1, e2, ··· , ek }. • Target: multi-dimensional Gaussian the same first two moments • k The two are poly( σ )-close in total variance distance (independent of n) Corollary: PTAS for approximate Nash Equilibrium in anonymous games.

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Computationally Efficient Learning Algorithm for PMD

Ilias Diakonikolas, Daniel M. Kane, Alistair Stewart • Learn any (n,k)-PMD with variance distance ϵ using size O˜(1/ϵ2) runs in time ˜(1/ϵ2) log n) Corollary: PTAS for approximate Nash Equilibrium in anonymous games with a constant number of strategies.

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning

The Game Theoretical Challenges in Machine Learning

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning New Frontiers

• Learn Dynamics, in the non-existence of Nash equilibrium. • Learn Adaptive environment with appropriate game theoretical setting. • Characterize Data in Equilibrium environment

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Some Restricted Subfields

• Spam filter • Stock market prediction • Credit card fraud dection

...... Game Theory in the Era of the Internet An Example of P2P Bandwidth Sharing Learn Auction ? Learning and Nash Equilibrium The Game Theoretical Challenges in Machine Learning Hello, Game Theory

Thank you, Machine Learning ......