Index

A G-atom, 297 Active radar, 1103 Attack strategy, 1188, 1207, 1214 Adaptive adversary, 1259 Attenuation level, 434, 437, 444 Adaptive dynamics, 477–484, 489, 490, Aubin, J.-P., 971 502, 503 Auction, 1187, 1197, 1204, 1206 asymmetric game, 502 Augmented proportional navigation, 1081 multi-dimensional, 489 Automatic generation control (AGC), 1189, Adaptive utility, 1260 1207, 1216, 1217, 1220 Additive reward and additive transitions Auxiliary problem, 438 (ARAT) game, 295 Availability, 1207 Adjoint equation, 645 Average Stackelberg cost, 43 Adjoint variable, 381 Averaging dynamics, 911–913 Admissible control, 163, 195 Avoidance, 1001 Advertising games, 869–875 Aggregative game, 589 Agreement, 637, 647–649 B Aircraft Ba¸sar, T., 1003 control, 957 Backward constructions, 953 landing, 957 Backward induction, 498 take-off, 957 Backward stochastic differential Airport security, 1250–1251 equations, 405 AK model, 180 Bankruptcy games, 829, 837–845 Algebraic Riccati equation, 1095 Barrier, see Barrier line Algebraic sum, see Minkowski sum Barrier line, 968, 970, 973, 977, 981, 986, 987, Altruism 990, 993, 995, 997 non-paternalistic, 327 Battery degradation, 1200 paternalistic, 323 Battle of the sexes, 1054 Amazon Mechanical Turk (AMT), 1257 Bayesian games, 55 Angular frequency, 1216, 1217 Bayesian Nash equilibrium, 56 Anonymous sequential games, 351 Bayesian Stackelberg game, 1229, Anti-Bourgeois, 1056 1250, 1254 Approachable set, 267, 268 Behavioral models, 1244, 1257, 1259, 1266 Approximating sets, 756 Behavioral Stackelberg equilibrium, 35 ARMOR, 1225, 1228, 1236, 1251 Behavior strategy, 750 ASPEN, 1237–1239 Belief-free equilibrium, 754 Asymmetric games, 1053 Bellman equations, 173, 176, 196, 203, Asymptotically stable, 1049 207, 646 Asymptotic uniform value, 271 Ben-Asher, J.Z., 1029 Asymptotic value, 255, 266 Bequest games, 742, 761 Asynchronous model, 921 Bernhard, P., 953, 956, 992

© Springer International Publishing AG, part of Springer Nature 2018 1271 T. Ba¸sar, G. Zaccour (eds.), Handbook of Dynamic Game Theory, https://doi.org/10.1007/978-3-319-44374-4 1272 Index

Bertrand competition, 781, 784, 786, 788, Chain store game, 496 801–803, 808, 810, 815 entry deterrence game, 497 Bertrand equilibrium, 787, 788, 801–803 Channel coordination, 882–884 Best response, 351, 355, 358, 1053, 1064, Characteristic function, 602, 603, 605 1068, 1194 ˛-characteristic function, 603 -Best response, 568 ˇ-characteristic function, 603 Better reply path, 524 ı-characteristic function, 604 Better response process, 534 -characteristic function, 603 Biased proportional navigation, 1103 Characteristics, 977 Bias injection attack, 1218 Cauchy, 963 Big Match game, 253 Charging pattern, 1199 Bimatrix game, 194, 195, 198, 1059 Charging rate, 1200 Bishop-Cannings theorem, 468 Cheating period, 647 Blackout, 1207 Chentsov, A.G., 956, 970 Blackwell condition, 267 Chernous’ko, F.L., 957 Blackwell game, 251 Climate change, 704, 709, 710, 720 Bolza problem, 377, 385 Closed-loop, 436, 437 Bonds, 831 information structure, 163 Bonds and embedded options, 831–832 Stackelberg equilibrium, 198 Borda rule, 936, 937, 939, 941 Close-knit family, 580, 581 Bottom-up approach, 360 r-Close-knit, 581 Bounded confidence, 912, 920–924 Clustering, 911, 947, 1218, 1219 Bounded rationality, 1226, 1232, 1233, 1235, Coalition, 1204, 1205 1236, 1243, 1245, 1249, 1256–1262 Coalitional game, 1204, 1205 Bounded rationality models, 1258–1259 Coalition formation, 910, 944–947, 1199, Bounded surveillance, 1226, 1232, 1236, 1205, 1206 1262–1264 Collision, 1000 Bounded surveillance model, 1262–1264 Collision course, 1082 Bourgeois, 1056 Collusive equilibrium, 102–108, 201, 203, Braess’ paradox, 1160 205, 208 Breakwell, J.V., 958, 983, 989 Command signal, 1012 Browder’s fixed point theorem, 313 Commit, 164 Brownian motion, 1201 Commitment-short-term, 184 Bushaw, D.W., 952 Common noise, 366 Communication networks, 368, 1144 Comparison principle, 394, 402 C Competitive contagion game, 573, 575–577 Caching game, 549 Competitive market, 1193, 1195 uncapacitated, 562, 565, 569 Complete information game, 6, 56 Callable/redeemable bonds, 831 Computational methods, 16 Candidate stability, 943, 944 Computing correspondences, 757, 765 Capacitated selfish replication game, 565, Condition 567–570 irreducibility, 307 Capitalists, 200, 202, 212 strict diagonal dominance, 286 Caplow’s theory, 946 strong stochastic dominance, 299 Car, 960, 961 Conditional Pareto efficiency, 662 Dubins’, 961, 1003 Condorcet criterion, 936, 938 Reeds-Shepp’s, 962, 995, 1003 Confidence bound, 921, 924 Carathéodory theorem, 284 Conformity, 913, 920, 926, 934 Cardaliaguet, P., 971, 991 Congestion control, 1146 CASS, 1241–1243 Congestion game, 549–553, 555, 556, 559, 589 Causality, 436, 448 asymmetric network, 555 Cell problem, 424 definition, 550 Cesaro limit, 161 in market sharing, 555–559 Index 1273

network, 551, 553 Core, 596, 605, 606, 616, 622 player-specific, 555 Corporate games, 829, 845–859 pure-strategy NE, 553–555 Corrector, 424 symmetric network, 553–555 Correlated equilibrium (CE), 18–20, 519 Conjectural variations, 782–784, 794 Correspondence, 115, 730 Consensus algorithm, 515 lower measurable, 218 Consistency condition, 363 upper semicontinuous, 218 Constant injection attack, 1218 weakly measurable, 218 Contingent claim, 829 Co-state variable, 645, 647 Continuation payoff, 173, 190 Cost function(al), 4, 9, 11, 14, 21, 26, 35, Continuous-kernel games, 5, 36 38–40, 43, 45, 50, 51, 55, 1189, formulation, existence and uniqueness, 1199–1201, 1214, 1215 36–38 Cost learning, 876 stability and computation, 38–40 Cost measure, 439, 453 and Stackelberg equilibria, 40–44 Coupled constraint, 114 Continuously stable strategy (CSS) Coupled constraint equilibrium (CCE), concept, 476 114–116 and adaptive dynamics, 477–479 Coupled-constraint problem (CCP), 134 asymmetric game, 503 Coupled-reaction mapping, 115 multi-dimensional, 489 Coupled state-constraints Continuous opinion dynamics, coupled dynamic game, equilibria for, 913–924 136–138 Continuous trait space, 476 discounted case, 143–152 asymmetric game, 502 global change game, 138–143 one dimensional, 476 Hamiltonian systems, 119, 124 Contradictory criteria, 937 model and basic hypotheses, 133–135 Control m-person games, 114, 119 adaptive, 958, 1023 open-loop differential games, 124–133 extremal, 979 steady-state normalized equilibrium feedback, 953, 968, 1026 problem, 135 inputs, 1201 Cournot competition, 781, 783, 784, 786, 788, inseparable, 955 791, 794, 795, 799–801, 803, 808, open-loop, 954, 956 810, 812, 815, 817 separable, 955 Cournot equilibrium, 781, 782, 789, 791, 794, strategy, 1200–1202, 1214 795, 797, 799–804 Convenience cost, 1200 Cournot stochastic game, 302, 303 Convergence stable, 477 Crandall, M.G., 967 Convex hull, 1007, 1029 Credibility, 648 Convex optimization, 1212 Credible threats, 668 Cooperation, 1045 Critical tube, 1010, 1013, 1016–1018, Cooperation duration, 644 1021, 1023 Cooperative differential games, 600 Current-valued value function, 176 with random duration, 623–628 Curse of dimensionality, 178 Cooperative dynamic games, 634–668 Cyber attacks, 1207, 1212–1215, 1217 Cooperative games, 4, 634–668, 706, 718, 720, Cyber-physical power system, 1187, 726, 1199, 1203 1215, 1216 Cooperative outcome, 636 Cyber-physical system, 1187, 1212, 1213 Cooperative strategies, 657–661 Cyber security games, 1234–1235 Cooperative trajectory, 636, 650, 654, 661 Coordination games, 913 D bounded confidence, 920–924 Darwinian dynamics, 482–488, 505 definition, 909 Data injection, 1207–1210, 1217, 1218 innovation spread, 927–935 Day-ahead market, 1195, 1209 stubborn individuals, 914–920 Decision theory, 433 1274 Index

Decomposed Optimal Bayesian Stackelberg Distribution, 1186–1189, 1197–1200, Solver (DOBSS), 1228–1230 1202–1206 DOBSS MILP, 1229 Disturbance attenuation, 1099–1106 DOBSS MIQP, 1229 Double oracle, 1239–1241 Defaultable game options, 832–833 Dove, 1042, 1056 Defender, 1000 DS-continuity, 290 Defense strategy, 1188, 1207, 1211–1215 Dubins, L., 961 Deffuant-Weisbuch (DW) model, 921 Duhamel principle, 365 Definability, 257 Dynamic(s) DeGroot model, 912 best response dynamics, 472 Delay function, 550, 554–556 compensator, 436, 442, 443 Delta-Nash equilibrium, 923 consistency, 635, 648 Demand learning, 876–877 game(s), 6, 8, 19, 21, 23, 28, 35, 55, Demand response, 1187 634–668 Demand-side management (DSM), 1187, game theory, 1188 1188, 1197, 1199, 1202 instability, 637 Deregulated market, 1187 linear, 956, 1003 Deterministic games, 5, 45–50, 53 monotone selection dynamics, 471 Stackelberg solution, 52–53 nonlinear, 954 team problems, 51 programming, 390, 398, 953, 1220 two-person zero-sum games, 50 programming principle, 1006 Diagonally strictly concave, 119, 127, 129, reaction functions, 185 137, 141 simple motion, 952, 955–957, 960, Diet choice model, 1067 1000, 1006 Difference equations, 158 stability, 1207 Differential games, 6, 8, 57, 639, 649–652, system, 158, 1188, 1207, 1212–1214, 1216 704, 706–708, 868, 874, 878, Dynkin game, 332 880, 881, 952, 953, 1200, 1201, 1213, 1215 with hard bounds, 1089 E nonzero-sum, 62–108 Economic dispatch, 1190, 1217 zero-sum, 62, 105 Electricity demand, 1199, 1202 Diffusion game, 549, 570, 573, 582, 589 Electricity prices, 1187, 1198, 1202, 1206 competitive contagion in networks, Electric vehicles, 1187, 1197, 1199, 1203 573–578 Empirical measure, 350, 351, 362, 364 coordination-based, 578–582 Encryption, 1210 deterministic, 571–573 Endogenous growth model, 202 Dilution effect, 833 Energy market, 1187, 1189, 1191, 1208 Dimensionless variables, 961 Entropic risk measure, 331 Direct reciprocity, 1045 Entry deterrence (accommodation), 782, 818 Discounted criterion, 161 Envelope condition, 185 Discounted payoff, 1215 Equilibrium, 115, 116, 118 Discrete control scheme, 955, 968, correlated, 317 995, 1005 correlated with public signals, 291 Discrete discriminating kernel, 971 Nash constrained, 321 fully, 972 robust Markov perfect, 331 Discrete opinion dynamics, 924–935 selection, 535–541 Discrete-time analysis, 653–662 stationary almost Markov, 288 Discriminating domain, 409 stationary Markov, 288 Discrimination, 954, 1005 subgame-perfect, 288 Dispersal, 1064 Erdos-Renyi graph, 572, 573 Distributed generation (DG), 1187, 1197, Ergodicity, 244 1198, 1204 Escape times, 1095 Distributed storage, 1187 ESS, see Evolutionarily stable strategy (ESS) Index 1275

EV, 1199–1203 Ferry protection, 1241 Evader, 952, 954, 955, 957, 983, 990, 992, Fictitious play (FP), 18, 531–532 1000–1002, 1011–1013, 1026 joint strategy, 532–533 Evolutionarily stable strategy (ESS), 462, 463, variants of, 532 467, 1042, 1049 Filippov theorem, 284 candidate, 487 Finance, dynamic games, 828 and invasion dynamics, 464–465 Finance game-theoretic models, 828 and replicator equation for matrix games, Finite-escape time, 1087 465–467 Finite games, 5 2-species, 492, 1054, 1064 existence and computation of weak, 1055, 1070 NE, 14–20 Evolutionary branching, 479 and Stackelberg equilibria, 28–36 Evolutionary game(s), 724–726 Finite improvement path, 553 distributed control design (see Game Finite population, 360 theoretic learning, distributed Fishery protection, 1244 control) Fishery’s management model, 162 dynamics, 471, 505 Fish war game, 298 Exceptional line, see Singular line Fixed point, 116, 1202, 1215 Exhaustible resources, 674, 685 Fixed-point condition, 115 antibiotics effectiveness, 692–694 Fokker-Planck-Kolmogorov (FPK) equation, extraction under different market 347, 358, 359, 361, 365, 1202 structures, 686 Folk theorem, 462, 467 under common access, 691–692 asymmetric games, 492 Exogenous disturbances, 1213 evolutionary game theory, 462 Exponential discounting, 188 Follower, 28, 191, 192 Extensive form, 21–23 Foraging game, 1066 Extensive form correlated Forest protection, 1256 equilibrium, 754 Forward-backward stochastic differential Extensive form game, 496, 1067 equation, 364 generic, 500 Fossil resources and pollution, 690–691 Extremal aiming, 995 FPK equation, see Fokker-Planck-Kolmogorov Extremal shift, 995, 1026 (FPK) equation Extremal trajectory, 979 Frequency relay, 1217, 1218 Front, 978, 981, 985, 990 Full Pareto efficiency, 662 F Function Fair choice, 936–940 universally measurable, 218 False target, 1002 upper semianalytic, 218 Federal Air Marshals Service (FAMS), 1225 Functional response, 1066 Feedback equilibrium, 198, 781, 783–786, Fundamental Cauchy matrix, 1004 789, 791, 794, 795, 797, 801, 802, Fundamental diagram of traffic, 1110, 1114 804, 806, 810, 811 Feedback games of exploitation, 677–681 Feedback information structure, 164, 175, G 196, 208 Game Feedback-Nash equilibrium, 177, 178, 191, absorbing, 256, 264, 315 199, 202, 203, 205, 208, 636, acoustic, 990, 992–994 646, 653 approximating, 1006 Feedback Stackelberg equilibrium (FSE), 91, ARAT, 295 94–97, 101 Big Match, 253 Feedback strategy(ies), 164, 194, 197, Blackwell, 251 210, 640 with complete information, 955 Feedback threat, 200 Cournot stochastic, 301–303 Feldbaum, A.A., 952 of degree, 968 1276 Index

Game (cont.) Nash equilibrium (see Nash equilibrium differential, 952 (NE)) linear, 954, 1003, 1028 non-cooperative games and equilibria, 8–13 one-dimensional, 1028 origins and classifications, 4–8 Dynkin, 332 quadratic games, 45–55 fish war, 298 social networks (see Social networks) group pursuit, 957 Gauss-Seidel iteration, 47, 49 homicidal chauffeur, 952, 958, 976, 1001 Generalized Nash equilibrium (GNE), 114, with incomplete information, 955 1211, 1212 isotropic rockets, 1001 Generation, 1186, 1187, 1189–1195, 1197, of kind, 968 1203, 1204, 1209, 1217, 1218 lifeline, 955 Geometric difference, see Minkowski limit-average stochastic, 307 difference linear-quadratic, 1028 G-function, 483 noisy stochastic, 296 fitness generating function, 483 perfect information, 262 Global agreement, 911 pursuit-evasion, 957, 1002 Global asymptotic stability (GAS), 122, 123, quitting, 316 144, 153 recursive, 252, 257, 264, 266, 317 Global reaction function, 115 resource extraction, 300 Goodwill models, 874–875 supermodular, 285, 303 Gossip model, 913 supermodular stochastic, 304 Graphical coordination games, 526, 541 surveillance-evasion, 989–991 Graphical games, 549 time-optimal, 953, 958 Great Fish War, 162 of two cars, 1001 Green security games, 1232–1233 zero-sum, 953 Group optimality, 634, 637–639 Game options, 830 Group selection, 1043 numerical approaches, 834 Growth model, 227–229 Games in normal (strategic) form, 67, 69 Growth rate, 200, 202, 212 Games of kind, 407 Guaranteed performance, 433, 455 Game theoretic learning, distributed control action profile, 518 distributed routing, 525–526 H equilibrium selection, 530, 535–541 Habitat selection game, 473, 1060 game design, 514, 517, 528–529 two-species, 493 game structures, 523–525 Hamiltonian, 167, 379, 401, 402, 426, 645, graphical coordination games, 526 955, 965, 973 human-agent collaborative systems, 518 Hamiltonian systems, 381 joint distribution, 519 competitive models, 123–124 learning design, 515, 517 Ramsey problem, 119–123 measures of efficiency, 520–521 Hamilton-Jacobi-Bellman (HJB) equation, model-based learning, 529, 530 75–76, 95, 347, 356, 359, 361, 363, performance guarantees, 517 365, 1202 robust learning, 530, 534–535 Hamilton-Jacobi equation, 953 setup, 513–514 Hamilton-Jacobi-Isaacs (HJI) equation, 380, smoothness, 522–523 397, 399, 403, 412, 419, 421, strategy profile, 519 426, 1215 universal learning, 530, 541–544 second-order, 401 vehicle target assignment problem, 527 Hamilton rule, 1047 Game theory, 1188, 1189, 1199, 1203, 1219 Handicap principle, 1071 in extensive form, 21–23 Hausdorff distance, 756, 757 hierarchical finite games and Stackelberg Hawk, 1042, 1056 equilibria, 28–36 Hawk-Dove game, 1042 incomplete information, 55–57 Hegselmann-Krause (HK) model, 920, 921 Index 1277

Historical evolution of game theory, Informational non-uniqueness, 8, 23 see Game theory Infrastructure security games, 1231 History of the game, 160, 163, 164 Innovation spread, 927–935 Hit-to-kill, 1087 Instant run-off, 935, 937, 939 HJB equation, see Hamilton-Jacobi-Bellman Integrity, 1207, 1208 (HJB) equation Intelligent Randomization In Scheduling Homicidal chauffeur, 952, 958, 960–1003 (IRIS), 1225, 1236–1237 Homicidal chauffeur game, 376 International environmental agreements, Homing missile guidance, 1083 716–726 Homogenization, 426 International pollution control, 708–716 Homotopy, 313 Internet, 1210 Horizontal queues, 1110 Intra-personal dynamic game, 189 Hotelling Rule, 685 Invariant distribution, 327 Human subjects experiments, 1259, 1262 Invasion dynamics, 464 Hybrid hierarchical equilibrium, 1212 asymmetric games, 492 Hyperbolic discounting, 159, 734, 738, 758 Involute, 975 Hyperbolic preferences, 329 Ionescu-Tulcea Theorem, 747, 751 Hypercube, 572 Irrelevant alternatives, 939 Isaacs, R., 375, 378, 407, 952, 958, 961, 964, 976, 1001 I Isaacs’ condition, 379, 397, 399, 403, Ideal Free Distribution (IFD), 473, 1060 412, 955 two-species, 495 Isaacs’ equation, 384 Implicit equilibrium solution, 147 Isaacs’ method, 963, 969 Implicit Nash equilibrium, 148–149 Isochrone, see Front Implicit steady state equilibrium problem, Iterated Prisoner’s Dilemma, 1045 146–148 Impulses, 454–457 Imputation, 602, 603, 605, 607 J Imputation distribution procedure (IDP), Jacobi iteration, 47, 49 610, 612 Jamming, 1104 Incentive equilibrium, 599 Jamming attack, 1003 Incomplete information game, 6, 55–57 Jankov-von Neumann theorem, 220 Increasing patience, 188 Joint payoff, 118 Incremental strategy generation, 1237–1239 Joint strategy fictitious play (JSFP), 532–533 Independence of irrelevant alternatives (IIA), 939 Indirect reciprocity, 1045 K Indirect utility, 328 Kakutani fixed-point theorem, 115 Individual player’s payoff, 638–639, 643–644 Kalai-Smorodinsky bargaining solution, 647 Individual rationality, 596, 597, 599, 603, 637 Kalman, 446 Industrial organizations, 184 Kalman filter, 1102 Inefficiency of NE, 18 Kantorovich-Rubinstein (KR) metric, 361 Infinite game, 5, 36–40 Karush-Kuhn-Tucker multipliers, 116–117 Infinite horizon concave differential Kelly mechanism, 1150–1152 games, coupled state-constraints, Kin selection, 1045, 1047 see Coupled state-constraints, 133 Krasovskii, N.N., 953, 954, 970 Infinite horizon problem, 377 Kruzhkov transform, 413 Infinite population, 360 Kuratowski Ryll-Nardzewski theorem, 218 Information, 435–438 patterns, 354–355 sets, 21, 23, 25, 27 L structure, 163 Lagrangian, 168 Informational inferiority, 23 Large finite populations, 347, 351 1278 Index

Lasry-Lions monotonicity condition, 361 M Latency function, 1110, 1112, 1114, 1139 Major agent, 353, 365 Lattice, 572, 581 Marauder, 1057 Leader, 5, 28, 29, 31, 34, 41, 43, 52, 192 Marginal contribution, 529 Leader’s Stackelberg cost, 31, 33, 36, Marginal cost, 1195 41, 43 Marketing, 866–900 Leadership Marketing channels, 881–891 channel, 885 Market power, 1197 domain, 409 Market sharing game, 555, 556, 558, 559 Learning, 1256–1262 uniform, 557 algorithm, 1206 Markov, A.A., 961 in games, 513, 515, 516 Markov control-state substitutability Least best response, 568 (complementarity), 781, 786, 788 Level set, 958, 1005 Markov decision process(es) (MDP), 347, 351, Levhari-Mirman over-exploitation, 678 1250, 1253–1255 Lévy processes, 351 Markov equilibria, 768, 771 Lewin, J., 953, 989 Markovian equilibrium, 175 L2-gain, 450 Markovian information structure, 164 Limiting behaviour, 326 Markovian strategy, 164 Limit of average criterion, 161 Markov-perfect Nash equilibrium (MPNE), Line 679, 688 barrier, 967, 968, 970, 977, 981, Markov process, 1213 985, 986 Markov-stationary strategy, 750 dispersal, 967, 977, 983, 987 Markov strategy, 356, 750 equivocal, 967, 977, 983 Master equation, 347, 361 focal, 983, 985, 986 Master slave approach, 1245 singular, 967, 977, 983 Matching, 1204–1206 switch, 983 Matrix game, 5, 14, 16, 22, 24–26, 43 universal, 967, 977, 983 Maximal stable bridge, 955, 958, 970 Linear complementarity problem, 312 Maximin Markov decision process, 221 Linear quadratic differential game(s) (LQDGs), Maximum principle, 167, 486 80–90, 1084, 1093 McKean-Vlasov equation, 350 guidance law, 1087 McKean-Vlasov stochastic differential Linear-quadratic games, 53, 177 equation, 347 Linear-quadratic systems, 358 Mean field, 354 Line-of-sight, 1084 equilibrium, 1146, 1171, 1202 Lion and man, 375 game, 1167, 1199–1202 Lions, P.-J., 967 Mean field game (MFG) theory LMP, 1193, 1194, 1196, 1209, basic PDE formulation, 360, 361 1211, 1212 common noise problem, 366 Load-frequency control, 1217 communication networks, 368 Load profile, 1199 congestion studies, 349 Load shedding, 1217, 1219 diffusion models, 349–351 Local asymptotic stability, 465 economics and finance, 348 Locally stable Nash equilibrium, 39 finite state models, 351 Locally superior, 468 individual agent performance functions, Locational marginal prices, 1193 noncooperative games, 352 Log linear learning, 535–537 information patterns, 354–355 binary, 537–539 linear-quadratic systems, 358 Long-run average (LRA) payoff, 680 major and minor agents, 365 Loop shaping, 442 Markov chains and discrete time Lotka-Volterra competition model, 487 processes, 351 Lower value, 10, 12, 51 master equation method, 361 Lyapunov function, 469 mean field type control optimality, 357 Index 1279

Nash equilibria, 355–356 Monitoring and threat strategies, 644–648 nonlinear Markov framework, 364–365 Monotonicity, 937 nonlinear Markov processes, 351 Multi-level selection, 1045 non-linear systems, 358–360 Multiple Nash equilibria, 705 opinion dynamics, 349 Multistage game, 158 Pareto optima, 356 Multistage game in a state space, 159 probabilistic approach, 363–364 Mumbai terrorist attack, 1239 residential power storage control, 367 risk sensitive performance functions, 352 N SDE, 363 Narrow throat, 958, 1016, 1018 social optima and welfare optimization, Nash bargaining solution, 637, 649, 656 356 Nash certainty equivalence (NCE), 347, 358 stochastic growth models, 369 Nash equilibrium (NE), 9, 22, 24, 68–90, 94, team optima, 357 102, 104, 106–107, 202, 203, 205, vaccination games, 349 208, 210, 346, 348, 354–356, 358, Measurable selector, 218 359, 363, 462, 467, 513, 922, 923, Measurement errors, 1208 942, 943, 945, 947, 957, 1042, 1068, Measurement residuals, 1209 1115, 1118, 1146, 1153, 1194, 1201, Measurement uncertainty, 1099 1211, 1220 p-Median problem, 587 asymmetric games, 492 Melikyan, A.A., 953, 956, 968 best Nash equilibrium, 1116, 1124 Memory strategies, 102–108 congested, 1120 Merge and split, 1206 correlated equilibrium, 18–20 Mergers, 781, 795, 796, 800, 801 cost, 1119 Merz, A., 953, 958, 983 essential uniqueness, 1120 Method of infinite/continuous-kernel games (Cauchy) characteristic, 964, 966 (see Continuous-kernel games) Isaacs’ (see Isaacs’ method) mixed strategy, 519, 575 programmed iterations, 956, 970 neighborhood strict NE, 480 Microgrid, 1189, 1198, 1199, 1203–1206 neutralization and domination, 15–16 MIDAS, 1225, 1244–1246 nonzero-sum finite games and Nash’s Miele, A., 957 theorem, 17–18 Min-cost flow, 554 number of equilibria, 1123 Minimal size coalition, 945–946 off-line computation of MSSPE, 16 Minimax, 1214 on-line computation of MSSPE and Minimax theorem, 7, 15–16, 220 MSNE, 18 Minkowski difference, 954, 1007 outcome, 22 Minkowski sum, 1007 perfect equilibrium, 25–27 Minor agent, 353, 365 proper equilibrium, 27 Mischenko, E.F., 957 pure-strategy, 518, 551, 553–555, Miss 562, 589 one-dimensional, 1029 single-link-free-flow, 1120, 1121 two-dimensional, 1012, 1028 solution, 640, 653, 663 Missile guidance, 1106 subgame perfect (SPNE), 498 Mixed Stackelberg equilibrium strategy, 33, support, 1119 34, 43 zero-sum finite games and minimax Mixed-strategy Nash equilibrium (MSNE), 11, theorem, 14–15 17, 18, 20, 26, 38 -Nash equilibrium, 356, 359 Mixed-strategy saddle-point equilibrium Nash–Pareto pair, 1054 (MSSPE), 11, 14, 16, 18, 38 Nash’s theorem, 17–18 Model-based learning, 530 Natural resources, 674 Model reduction, 424 exploitation, 674 Mode of the game, 208 Negative compensation attack, 1218 1280 Index

Neighborhood invader strategy (NIS), O 477, 480 Oblivious equilibria, 347 asymmetric game, 504 Observability, 1207, 1208, 1212 neighborhood invader strategy, 480 Observation, 438 and replicator equation, 479–482 Oligopolistic exploitation, 682 Neighborhood superior, 481 Oligopoly, 149–152 asymmetric game, 504 Olsder, G.-J., 953, 989 Neighborhood superiority, 481, 503 Online algorithm, 513 Network creation game, 582, 583, 585–587 Online learning in routing, 1109 local search for, 585–587 Open-loop, 436 PoA, 583–585 controls, 385 Network formation game, 549, 582, 584, equilibrium, 780–783, 786, 788, 588, 589 789, 791, 794, 795, 797, 806, coordination-based, 587–588 811, 812 local search for, 585–587 games of fishery, 676–677 PoA, 583–585 information structure, 164, 194, 196 Network games, 347 strategy, 67, 69, 91, 195 aggregative games, 589 Open-loop differential games (OLDG), congestion games, 550–559 124–133 diffusion games, 570–582 Open-loop Nash equilibrium (OLNE), 69, 167, network formation games, 582–588 170, 645 rent-seeking games, 589 necessary conditions for, 72 resource allocation games, 559–570 Open-loop Stackelberg equilibrium (OLSE), Network reciprocity, 1045 91–94, 198 Neutralization, 15–16 Operator norm, 434, 439 Newton’s second law, 1216 Opinion dynamics, 909 NIS, see Neighborhood invader strategy (NIS) averaging dynamics, 911–913 Nobel Prize, 7 continuous, 913–924 Non-atomic routing games, 1115, 1137 definition, 910 Non-compliant first strategy, 1118, 1124 discrete, 924–935 Non-cooperative games, 5, 352, 355–356, Opportunistic crime security games, 1194, 1199, 1211 1231, 1233 elements and equilibrium solution, 8–11 Optimal control, 159, 1213–1215 security strategies, 11–12 guidance law, 1087 strategic equivalence, 12–13 law, 1085, 1091–1093 Non-cooperative payoff, 637, 640, 646 Optimal dispatch, 1217 Non-dictatorship, 937 Optimal foraging, 1067 Nonlinear feedback strategies, 781, 796, 797, Optimal guidance law, 1082 801, 803 Optimality principle, 635, 647, 649, 656 Nonlinear Markov processes, 351, 364–365 Optimal power flow, 1190 Nonlinear programming, constrained, 310 Optimal response set, 30, 35 Non-minimal phase zero, 1103 Optimal strategy, 31, 42 Non-transferable utility (NTU), 634–668 Order transfer function, 1011 Nonzero-sum games (NZSGs), 5, 12, 13, Organization of Petroleum Exporting Countries 17–18, 24, 37, 39, 42, 50, 51 (OPEC), 685 Normal form, 167, 170 Outliers, 1209, 1211 Normal form games, 464 Overcompensation attack, 1217 asymmetric, 491 Over-frequency relays, 1217 bimatrix games, 491 Overtaking equilibrium, 124, 125, 127, 128, matrix games, 466 140–141 Normalized equilibrium, 117–119 Overtaking optimality criterion, 161 Not subgame perfect solution, 192 Owner intruder game, 1056 NP-hard, 556, 572, 583, 587 Own-state feedback Nash Numerical methods, 756–758 equilibrium, 1201 Index 1281

P Pontryagin, L.S., 953, 957 Pachter, M., 958 Pontryagin maximum principle, 381 Packet scheduling, 1144, 1146 Positional formalization, 954 Parallel network, 1115, 1135 Potential function, 551–554, 560, 569, 580, Pareto efficiency (PE), 102, 939 581, 922 Pareto optimality, 635, 644, 649, 655 exact, 551, 552, 555, 556, 560, 565, 579 Pareto optimal strategies, 202, 203, 206, 207, generalized ordinal, 565, 567, 568, 356, 637–638 586, 587 Parker’s matching principle, 1061 Rosenthal, 556 Parseval’s equality, 440 Potential game, 524, 589, 922, 924–926 Partial-cooperation, 705, 706, 716–718 exact, 551–553, 565, 579 3-Partitioning problem, 572 generalized ordinal, 565, 567, 586 Patent races, 171 Pourtallier, O., 956 PAWS, see Protection Assistant for Wildlife Power flow, 1190, 1191, 1208, 1217 Security (PAWS) Power injection, 1190, 1191, 1208 Payoff, 160, 161, 163, 169, 172, 177, 178, 638, Preemption, 789, 791, 813, 815, 817, 818 640, 643–644 Present bias, 188 approximating, 1006 Price anticipation, 1152–1158 expected discounted, 223, 288 Price decentralization, 118 limiting average, 261 Price decomposition principle, 118 limsup, 252 Price of anarchy (PoA), 520–522, 558, 559, normalised discounted, 254 564, 569, 575–577, 583–585, 587, quasiconvex, 1005 1131, 1155, 1163, 1181 ratio average, 243 Price of stability (PoS), 521, 1131, 1132, 1136 robust expected average per unit time, 233 Price signal, 1197, 1199 sample path, 249 Price taking, 1151 terminal, 1003 Pricing games, 875–880 time average, 243 Princess and monster, 376, 415 uniform equilibrium, 314 Principle Pedestrian, 960 (Pontryagin) maximum, 952, 965 Perfect equilibrium, 25, 26 transition, 953, 965 Perfect information game, 496 Prisoner’s dilemma, 1044 Perfect public equilibria, 746 Probability weighting, 1259, 1260 Performance criterion, 160 Productive capacity, 781, 782, 790, 791, Performance index, 1214 815–818 Person-by-person optimality, 51, 356 Programmed iterations, 956 Petrosyan, L.A., 953 Proper equilibrium, 27, 56 Phase angle, 1190, 1191, 1207, 1208 Property Phasor measurement unit, 1208 of local convexity, 1007 Photovoltaic cells, 1187 stability, 971, 1005 Piecewise deterministic control, 166 viability, 971 Pigovian network, 1163 Proportional navigation, 1081, 1087 Planning, 1262–1264 Prospect theory, 1256, 1260 Players set, 8, 21 Prosumers, 1187, 1188, 1198, 1199 PLS-complete, 553, 555 PROTECT, 1225 Plurality, 935, 940, 941 Protection Assistant for Wildlife Security Poaching, 1232, 1246, 1256, 1257 (PAWS), 1225, 1249, 1259 Point Pschenichnyi, B.N., 953 of local concavity, 979 Pseudo-gradient, 118, 152 of local convexity, 979 Pseudo-Hamiltonian system, 124, 127, 144 Point-to-set mapping, 115, 116 p-superior, 481 Policy games, 746, 765 asymmetric game, 504 Policy iteration, 259 Public goods game, 1048 Polyhedral approximation, 1006 Public goods provision, 662–668 1282 Index

Public perfect equilibrium, 748 Replicator equation, 462–464, 466, 467, Punishment, 1045 471–475, 479 Punishment period, 647 asymmetric games, 492 Purification, 331 continuous-time, 470 Pursuer, 952, 954, 955, 957, 983, 990, continuous trait space, 479 1000–1002, 1011, 1013 discrete-time, 469 fast, 1013–1015 doubly symmetric matrix games, 471 more agile, 995, 996, 998, 999 Hawk-Dove game, 1044 slow, 1017, 1018, 1023 for matrix games, 465–467 Pursuit-evasion games, 375, 378, 411 mixed-strategy, 469 NIS and, 479–482 one-dimension, 467 Q properties of, 467 Quadratic games, 45 pure-strategy, 469 deterministic games, 45 trajectories, 470 stochastic games, 53–55 Residential power storage control, 367 Quantal response (QR), 1243, 1245 Resource allocation game, 549, 559, 589 Quasi-hyperbolic discounting, 188, 736, 739, capacitated selfish replication game, 740, 758, 766, 771 565–570 Quincampoix, M., 971, 991 graphical, 559–561 uncapacitated selfish caching game, 562–565 R Resource extraction game, 639, 650 Ramsey optimal taxation problem, 733, 763, Restricted spatial adaptive play (RSAP), 765, 768 925–926 Ramsey problem, 119–123 Riccati differential equation, 86, 87, 89, 1102 Random duration, 623–628 Riccati equation, 177, 358, 446, 447, 449, 1085 Random matching game, 927 Risk dominant equilibrium, 581, 582 Reaction set, 192, 196 Road network security, 1239 Real-time market, 1195, 1209 Robust control, 1213 Real-time pricing, 1187, 1199 Robust distributed learning, 534–535 Real world evaluation, 1264–1266 Robust price of anarchy, 522 Recursive game, 252 Rock-Scissors-Paper (RSP) game, 469, 470 Reeds, J., 961 Routing game, 549, 550, 588, 589, 1158–1165 Regime-switching strategies, 684–685 Row-stochastic matrix, 911 Region RUGGED, 1239–1241 primary, 977 regular, 977 S secondary, 977 Saddle point, 1215 Regret matching, 530, 542 Saddle-point equilibrium (SPE), 9, 32, 50 Regular optimal trajectories, 1091 Saint-Pierre, P., 971, 991 Reinforcement learning, 1197 Satisfaction equilibrium, 1212 Renewable energy, 1187, 1203 Scalability, 1226, 1236–1249 Renewable generation, 1197 Scheduling game, 1166–1181 Renewable resources Security, 1188, 1189, 1206, 1207, 1210, exploitation of renewable natural assets, 1212–1214, 1216 675–681 Security games exploitation under oligopoly, 682 bounded rationality modeling and learning, tragedy of the commons, 675 1256–1264 Rent-seeking game, 589 cyber, 1234–1235 Repeated game, 158, 746 evaluation, 1264–1266 Repeated Stackelberg Security Game (RSSG), green, 1232–1233 1257, 1259 infrastructure, 1231 Replicator dynamics, 1049 opportunistic crime, 1233 Index 1283

scalability, 1236–1237, 1249 Social optima, 356, 1132 SSG, 1226–1230 Social welfare, 939–940, 1191, 1192, 1195 uncertainty, 1249–1255 Solution Self-generation, 748, 752 generalized, 967 Selten’s Theorem, 1053 minmax, 967 Semi-anonymous potential games, 540 viscosity, 967 Semi-Markov equilibrium, 754 Sophisticated voting, see Strategic voting Semipermeable curve, 968, 973, 981 Space interception, 958 of first type, 969, 995 Spatial adaptive play (SAP), 925, 926 of second type, 969, 995 Spectral radius condition, 1102 Sensitivity transfer function, 444 Spinning reserve, 1217 Sequential equilibria, 733, 757, 758, 764 S-shaped probability weighting function, 1260 Set Stability, 705, 716–720 Lebesgue (see Level set) Stable bridge, 409, 954, 1024 level, 985 maximal, 954, 958, 970, 1005, 1009 critical (see Critical tube) Stable Nash equilibrium, 39 reachable, 954, 1001, 1025 Stackelberg solvability, 954, 978, 1006 cost, 30, 31, 33, 34, 41, 43 target, 954 feedback, 194, 196–198 rectangle, 993 followers, 1116 terminal, 955 game, 191–193, 197, 1199 conic, 990 leader, 1116 weak-invariant, 1005 optimal strategy, 1117, 1124, 1128 Sex-ratio, 1052 routing game, 1109–1110, 1116 Shadow price, 117, 152 strategy, 1116 Shapley value, 596, 605, 606, 616 support, 1126 SHARP, 1259 Stackelberg equilibrium (SE), 30, 41, 91–102 Shepp, L., 961 and continuous-kernel games, 40–44 Shinar, J., 958, 1011, 1029 in mixed and behavioral strategies, 32–36 Simple game, 420 in pure strategies, 28–32 Simple motion, 952 Stackelberg security games (SSG), 1226–1230 Simultaneous moves, 183 Standard Model of Economic Behavior, 683 Single-peaked preferences, 938–939 Standard problem, 437 Singular perturbation, 426 State, 150 Singular surface, 953, 967 equation, 158 Sliding mode, 997 estimation, 1189, 1207, 1208 Smart distribution system, 1198 space, 158 Smart grid, 1186, 1188, 1199, 1203 State-feedback Nash equilibrium (SFNE), Smoothness, 522 70–71 SNARES, 1241 necessary conditions for, 72–73 Social choice, 909–910, 935–936 sufficient conditions for, 76 Condorcet criterion, 936, 938 State-feedback strategy (control), 67, 68 contradictory criteria, 937 Static behavior, 780, 786–791, 794–797, function, 935 799–804 single-peaked preferences, 938–939 Static game, 6, 22, 41, 42, 46, 53, 55 social welfare, 939–940 Stationary discounted games, 176 strategic candidacy, 942–944 Stationary strategy, 1215, 1220 strategic voting, 940–942 Stationary system, 161 theory of coalitions, 944–947 Steady state, 132, 135–138, 146–148, 153, Social graph, 914, 916, 918–920, 924, 926, 1191, 1214, 1217 927, 929, 934 equilibrium, 130–131 Social networks normalized equilibrium problem, 135 opinion dynamics, 909–935 Stochastically stable, 580, 588 social choice, 909–910, 935–947 Stochastic control, 1201 1284 Index

Stochastic game, 6, 53–55, 750, 1196, 1197, Subjective utility, 1245 1213, 1215, 1219, 1220 Subjective utility quantal response (SUQR), Stochastic growth models, 369 1244, 1245 Stochastic kernel, 219 Submodular, 522 Stochastic maximum principle, 347, 357, 363 Successive elimination, 936 Stochastic process, 435, 454, 457 Sunspot equilibrium, 754 Strategic agents, 171 Support function, 1006 Strategic candidacy, 910, 942–944 Swallow tail, 979 Strategic dynamic programming, 731–733, Switch envelope, 985 737, 741, 746–756, 758–765 Symmetric parallel congestion games, 540 Strategic equivalence, 12–13, 79, 85 Synchronous frequency, 1216 Strategic voting, 910, 940–942, 947 Synchronous generator, 1212, 1216 Strategy, 6, 67, 164, 385 Synchronous model, 921 behavioral, 22 System operator, 1187, 1191, 1197, 1211 Borel measurable, 236 correlated mixed (CMS), 20 delay, 386 T feedback, 378 Tarski fixed point theorem, 753 mixed, 10 Tauberian theorem, 257 for nature, 330 Team optima, 357 nonanticipative, 386 Team-optimal solution, 51 nonanticipative strategy with delay, 386 Team problem, 12, 24, 51 parallel approach, 955 Terminal time, see Termination instant pathwise nonanticipative, 404 Termination instant, 1003, 1012, 1028 positional, 954, 1005 Theorem pure, 10 Browder’s fixed point, 313 random, 418 Carathéodory, 284 random delay-strategy along a Filipov, 284 partition, 396 Jankov-von Neumann, 220 security, 11 Kuratowski Ryll-Nardzewski, 218 semistationary, 236 minimax, 220 set (space), 21 Tauberian, 257 stationary almost Markov, 287 The Rock–Paper–Scissors game, 1049 stationary Markov, 287 Threat, 159, 165 universally measurable, 236 Time consistency, 101, 596, 597, 599, 600, Strict diagonal concavity-convexity assumption 619, 621, 622, 628, 635, 649, 650, (SDCCA), 128, 129 733, 741, 758, 762 Strongly time consistent, 94, 96, 101 of cooperative solution, 606–619 Strong Nash equilibrium, 945 Time consistent, 71, 101–102, 171 Strong stability, 469 Time constant, 1011 Strong Stackelberg Equilibrium (SSE), Time inconsistency, 322, 329 1228, 1236 Time-scale separation, 1214 Strong time consistency, 597, 620, 622 Tit for Tat (TFT), 1045 Stubbornness, 914, 919, 920 Top-down approach, 360 Sub-game perfect, 356 Total latency, 1116 Sub-modular function, 559 Trading period, 1200 Subbotin, A.I., 954, 967 Traffic routing, 1146 Subgame consistency, 648–652, 656–662 Trajectory shaping, 1093 Subgame consistent solutions, 649, 650, 656, Trajectory tubes, 435 661–662 Transaction costs, 453–455 Subgame perfect, 71, 94, 101, 104, 108, 210 Transferable utility, 1205 Subgame perfect equilibrium, 165, 175, 177, Transfer payments, 635, 649 203, 208, 209, 212 Transition rate, 270 Subgame perfection, 170–172, 175 Transit systems security, 1252–1253 Index 1285

Transmission, 1186, 1187, 1190, 1191, 1195, Verification theorem, 170, 175, 380 1197, 1208, 1213 Viability theory, 435, 971 Transportation networks, 1108–1139 Victory domain, 408 Transversality conditions, 168 Virtual bidders, 1209 Trembling hand, 171 Virtual container, 1002 Trigger strategy(ies), 108, 199, 201, 598 Virtual power, 1209, 1210 True hyperbolic preferences, 188 Viscosity solution, 393, 402, 452, 453, TRUSTS, 1253 456, 457 Turnpike property, 121, 123, 127, 137, 138, subsolution, 393, 402 144–146 supersolution, 393, 402 Voltage, 1186, 1190, 1191, 1204, 1207, 1208 Voltage regulation, 1216 U Voter preference, 935, 936, 938, 940, 942, 943 Unanimity, 909, 942, 943 Voting procedure, 943 Uncapacitated selfish caching game, Voting scheme, 935, 936, 938, 940, 947 562–565 Uncertainty, 1249–1255 Uncoupled constraint problem W (UCP), 134 Wardrop equilibrium, 1146, 1161 Under-frequency relays, 1217, 1218 War of attrition, 1051 Undetermined coefficients method, 179 Warrants, 833 Uniform value, 254, 265, 266 Weak formulation, 364 Universal learning, 541–544 Weakly acyclic game, 525, 560, 561 Unstable Nash equilibrium, 39 Weak Pareto efficiency (WPE), 937–938 Upper value, 10, 11 Weighted least square, 1208 URAC, 1250, 1251 Weighted sum of payoffs, 638, 654, 658, 660 Usable part, 969 Weighted supremum norm, 224 Usable part of the boundary, 383 Welfare, 781, 795, 796, 800, 802, 803 US Coast Guard, 1225, 1241 Wide-area operation, 1187 Utility function, 160 Wiener process, 350, 352, 362, 366–369 Utility maximization, 1151 Wildlife poaching game, 1257 Wildlife protection, 1246–1247 Wildlife security, 1249 V Wind disturbance, 957 Vaccination games, 349 Wind turbine, 1187 Valid utility game, 523, 559 Workers, 200, 201 Value function, 172, 173, 175, 176, 203, 207, Worst case design, 433 356, 358, 379, 388, 396, 398, 401, 403, 411, 413, 418, 420, 953, 955, 958, 964, 968, 1003, 1005, 1202, Y 1205, 1215 Yaesh, I., 1029 Value iteration, 1215 Value of altruism, 1132, 1136 Variable payoff weights, 656, 668 Z Variable weights, 653–662 Zero-effort miss distance, 1090 VCG mechanism, 1147–1149 Zero miss distance, 1013 Vectogram, 1006, 1014 Zero-sum games (ZSGs), 5, 9, 13–15, 17, Vector Lyapunov function, 123 37, 50