Nash equilibrium
In game theory, a Nash equilibrium is a situation where no player could gain more by changing their own strategy (holding all other players' strategies fixed) in a game.
≈ 16 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
In game theory, a Nash equilibrium is a situation where no player could gain more by changing their own strategy (holding all other players' strategies fixed) in a game. A Nash equilibrium is the most commonly used solution concept for non-cooperative games.
If each player has chosen a strategy — an action plan based on what has happened so far in the game — and no individual player can increase their own expected payoff by changing their strategy while the other players keep theirs unchanged, then the current set of strategy choices constitutes a Nash equilibrium. If two players Alice and Bob choose strategies A and B, (A, B) is a Nash equilibrium if Alice has no other strategy available that does better than A at maximizing her payoff in response to Bob choosing B, and Bob has no other strategy available that does better than B at maximizing his payoff in response to Alice choosing A. In a game in which Carol and Dan are also players, (A, B, C, D) is a Nash equilibrium if A is Alice's best response to (B, C, D), B is Bob's best response to (A, C, D), and so forth.
The concept of a Nash equilibrium dates back to the time of Cournot, who in 1838 applied it to his model of competition in an oligopoly. John Nash showed that there is a Nash equilibrium, possibly in mixed strategies, for every finite game.
Applications
Game theorists use Nash equilibrium to analyze the outcome of the strategic interaction of several decision makers. In a strategic interaction, the outcome for each decision-maker depends on the decisions of the others as well as their own. The insight underlying Nash's idea is that one cannot predict the choices of multiple decision makers if one analyzes those decisions in isolation.
Instead, one must ask what each player would do taking into account what the player expects the others to do. A Nash equilibrium is achieved when no player can improve their outcome by changing their decision, assuming the other players' choices remain unchanged.
The concept has been used to analyze hostile situations such as wars and arms races (see prisoner's dilemma), and also how conflict may be mitigated by repeated interaction (see tit-for-tat). It has also been used to study to what extent people with different preferences can cooperate (see battle of the sexes), and whether they will take risks to achieve a cooperative outcome (see stag hunt).
It has been used to study the adoption of technical standards, and also the occurrence of bank runs and currency crises (see coordination game). Other applications include traffic flow (see Wardrop's principle), how to organize auctions (see auction theory), the outcome of efforts exerted by multiple parties in the education process, regulatory legislation such as environmental regulations (see tragedy of the commons), natural resource management, analysing strategies in marketing, penalty kicks in football (i.e., soccer; see matching pennies), robot navigation in crowds, energy systems, transportation systems, evacuation problems and wireless communications.
History
Nash equilibrium is named after American mathematician John Forbes Nash Jr. The same idea was used in a particular application in 1838 by Antoine Augustin Cournot in his theory of oligopoly. In Cournot's theory, each of several firms choose how much output to produce to maximize its profit. The best output for one firm depends on the outputs of the others.
A Cournot equilibrium occurs when each firm's output maximizes its profits given the output of the other firms, which is a pure-strategy Nash equilibrium. Cournot also introduced the concept of best response dynamics in his analysis of the stability of equilibrium. Cournot did not use the idea in any other applications, however, or define it generally.
The modern concept of Nash equilibrium is instead defined in terms of mixed strategies, where players choose a probability distribution over possible pure strategies (which might put 100% of the probability on one pure strategy; such pure strategies are a subset of mixed strategies). The concept of a mixed-strategy equilibrium was introduced by John von Neumann and Oskar Morgenstern in their 1944 book The Theory of Games and Economic Behavior, but their analysis was restricted to the special case of zero-sum games.
They showed that a mixed-strategy Nash equilibrium will exist for any zero-sum game with a finite set of actions. The contribution of Nash in his 1951 article "Non-Cooperative Games" was to define a mixed-strategy Nash equilibrium for any game with a finite set of actions and prove that at least one (mixed-strategy) Nash equilibrium must exist in such a game.
The key to Nash's ability to prove existence far more generally than von Neumann lay in his definition of equilibrium. According to Nash, "an equilibrium point is an n-tuple such that each player's mixed strategy maximizes [their] payoff if the strategies of the others are held fixed. Thus each player's strategy is optimal against those of the others."
Putting the problem in this framework allowed Nash to employ the Kakutani fixed-point theorem in his 1950 paper to prove existence of equilibria. His 1951 paper used the simpler Brouwer fixed-point theorem for the same purpose. Game theorists have discovered that in some circumstances Nash equilibrium makes invalid predictions or fails to make a unique prediction.
They have proposed many solution concepts ('refinements' of Nash equilibria) designed to rule out implausible Nash equilibria. One particularly important issue is that some Nash equilibria may be based on threats that are not 'credible'. In 1965 Reinhard Selten proposed subgame perfect equilibrium as a refinement that eliminates equilibria which depend on non-credible threats.
Other extensions of the Nash equilibrium concept have addressed what happens if a game is repeated, or what happens if a game is played in the absence of complete information. However, subsequent refinements and extensions of Nash equilibrium share the main insight on which Nash's concept rests: the equilibrium is a set of strategies such that each player's strategy is optimal given the choices of the others.
Definitions
A strategy profile is a set of strategies, one for each player. Informally, a strategy profile is a Nash equilibrium if no player can do better by unilaterally changing their strategy. To see what this means, imagine that each player is told the strategies of the others.
Suppose then that each player asks themselves: "Knowing the strategies of the other players, and treating the strategies of the other players as set in stone, can I benefit by changing my strategy?" For instance if a player prefers "Yes", then that set of strategies is not a Nash equilibrium.
But if every player prefers not to switch (or is indifferent between switching and not) then the strategy profile is a Nash equilibrium. Thus, each strategy in a Nash equilibrium is a best response to the other players' strategies in that equilibrium.
Formally, let S_i be the set of all possible strategies for player i , where i = 1, \ldots, N . Let s^* = (s_i^*, s_{-i}^*) be a strategy profile, a set consisting of one strategy for each player, where s_{-i}^* denotes the N - 1 strategies of all the players except i . Let u_i(s_i, s_{-i}^*) be player i s payoff as a function of the strategies. The strategy profile s^* is a Nash equilibrium if u_i(s_i^*, s_{-i}^*) \geq u_i(s_i, s_{-i}^*)\ \text{for all}\ s_i \in S_i. A game can have more than one Nash equilibrium.
Even if the equilibrium is unique, it might be weak: a player might be indifferent among several strategies given the other players' choices. It is unique and called a strict Nash equilibrium if the inequality is strict so one strategy is the unique best response: u_i(s_i^*, s_{-i}^*) > u_i(s_i, s_{-i}^*)\ \text{for all}\ s_i \in S_i, s_i \neq s_i^*.
The strategy set S_i can be different for different players, and its elements can be a variety of mathematical objects. Most simply, a player might choose between two strategies, e.g. S_i = \{\text{Yes}, \text{No}\}.
Or the strategy set might be a finite set of conditional strategies responding to other players, e.g. S_i = \{\text{Yes} \mid p = \text{Low}, \text{No} \mid p = \text{High}\}. Or it might be an infinite set, a continuum or unbounded, e.g. S_i = \{\text{Price}\} such that \text{Price} is a non-negative real number.
Nash's existence proofs assume a finite strategy set, but the concept of Nash equilibrium does not require it.
Proof of existence
r_i(\sigma_{-i}) = \mathop{\underset{\sigma_i}{\operatorname{arg\,max}}} u_i (\sigma_i,\sigma_{-i}) Here, \sigma \in \Sigma , where \Sigma = \Sigma_i \times \Sigma_{-i} , is a mixed-strategy profile in the set of all mixed strategies and u_i is the payoff function for player i. Define a set-valued function r\colon \Sigma \rightarrow 2^\Sigma such that r = r_i(\sigma_{-i})\times r_{-i}(\sigma_{i}) . The existence of a Nash equilibrium is equivalent to r having a fixed point.
Kakutani's fixed point theorem guarantees the existence of a fixed point if the following four conditions are satisfied. \Sigma is compact, convex, and nonempty. r(\sigma) is nonempty. r(\sigma) is upper hemicontinuous r(\sigma) is convex. Condition 1. is satisfied from the fact that \Sigma is a simplex and thus compact.
Convexity follows from players' ability to mix strategies. \Sigma is nonempty as long as players have strategies. Condition 2. and 3. are satisfied by way of Berge's maximum theorem. Because u_i is continuous and compact, r(\sigma_i) is non-empty and upper hemicontinuous.
Condition 4. is satisfied as a result of mixed strategies. Suppose \sigma_i, \sigma'_i \in r(\sigma_{-i}) , then \lambda \sigma_i + (1-\lambda) \sigma'_i \in r(\sigma_{-i}) . i.e. if two strategies maximize payoffs, then a mix between the two strategies will yield the same payoff. Therefore, there exists a fixed point in r and a Nash equilibrium.
When Nash made this point to John von Neumann in 1949, von Neumann famously dismissed it with the words, "That's trivial, you know. That's just a fixed-point theorem." (See Nasar, 1998, p. 94.)
We have a game G=(N,A,u) where N is the number of players and A = A_1 \times \cdots \times A_N is the action set for the players. All of the action sets A_i are finite. Let \Delta = \Delta_1 \times \cdots \times \Delta_N denote the set of mixed strategies for the players.
The finiteness of the A_i s ensures the compactness of \Delta . We can now define the gain functions. For a mixed strategy \sigma \in \Delta , we let the gain for player i on action a \in A_i be \text{Gain}_i(\sigma,a) = \max \{0, u_i(a, \sigma_{-i}) - u_i(\sigma_{i}, \sigma_{-i})\}.
The gain function represents the benefit a player gets by unilaterally changing their strategy. We now define g = (g_1,\dotsc,g_N) where g_i(\sigma)(a) = \sigma_i(a) + \text{Gain}_i(\sigma,a) for \sigma \in \Delta, a \in A_i . We see that \sum_{a \in A_i} g_i(\sigma)(a) = \sum_{a \in A_i} \sigma_i(a) + \text{Gain}_i(\sigma,a) = 1 + \sum_{a \in A_i} \text{Gain}_i(\sigma,a) > 0.
Next we define: \begin{cases} f = (f_1, \cdots, f_N) : \Delta \to \Delta \\ f_i(\sigma)(a) = \frac{g_i(\sigma)(a)}{\sum_{b \in A_i} g_i(\sigma)(b)} & a \in A_i \end{cases} It is easy to see that each f_i is a valid mixed strategy in \Delta_i . It is also easy to check that each f_i is a continuous function of \sigma , and hence f is a continuous function. As the cross product of a finite number of compact convex sets, \Delta is also compact and convex.
Applying the Brouwer fixed point theorem to f and \Delta we conclude that f has a fixed point in \Delta , call it \sigma^* . We claim that \sigma^* is a Nash equilibrium in G . For this purpose, it suffices to show that \forall i \in \{1, \cdots, N\}, \forall a \in A_i: \quad \text{Gain}_i(\sigma^*,a) = 0.
This simply states that each player gains no benefit by unilaterally changing their strategy, which is exactly the necessary condition for a Nash equilibrium. Now assume that the gains are not all zero. Therefore, \exists i \in \{1, \cdots, N\}, and a \in A_i such that \text{Gain}_i(\sigma^*, a) > 0 .
Then \sum_{a \in A_i} g_i(\sigma^*, a) = 1 + \sum_{a \in A_i} \text{Gain}_i(\sigma^*,a) > 1. So let C = \sum_{a \in A_i} g_i(\sigma^*, a). Also we shall denote \text{Gain}(i,\cdot) as the gain vector indexed by actions in A_i . Since \sigma^* is the fixed point we have: \begin{align} \sigma^* = f(\sigma^*) &\Rightarrow \sigma^*_i = f_i(\sigma^*) \\ &\Rightarrow \sigma^*_i = \frac{g_i(\sigma^*)}{\sum_{a \in A_i} g_i(\sigma^*)(a)} \\ [6pt] &\Rightarrow \sigma^*_i = \frac{1}{C} \left (\sigma^*_i + \text{Gain}_i(\sigma^*,\cdot) \right ) \\ [6pt] &\Rightarrow C\sigma^*_i = \sigma^*_i + \text{Gain}_i(\sigma^*,\cdot) \\ &\Rightarrow \left(C-1\right)\sigma^*_i = \text{Gain}_i(\sigma^*,\cdot) \\ &\Rightarrow \sigma^*_i = \left(\frac{1}{C-1}\right)\text{Gain}_i(\sigma^*,\cdot). \end{align} Since C > 1 we have that \sigma^*_i is some positive scaling of the vector \text{Gain}_i(\sigma^*,\cdot) .
Now we claim that \forall a \in A_i: \quad \sigma^*_i(a)(u_i(a_i, \sigma^*_{-i}) - u_i(\sigma^*_i, \sigma^*_{-i})) = \sigma^*_i(a)\text{Gain}_i(\sigma^*, a) To see this, first if \text{Gain}_i(\sigma^*, a) > 0 then this is true by definition of the gain function. Now assume that \text{Gain}_i(\sigma^*, a) = 0 .
By our previous statements we have that \sigma^*_i(a) = \left(\frac{1}{C-1}\right)\text{Gain}_i(\sigma^*, a) = 0 and so the left term is zero, giving us that the entire expression is 0 as needed. So we finally have that \begin{align} 0 &= u_i(\sigma^*_i, \sigma^*_{-i}) - u_i(\sigma^*_i, \sigma^*_{-i}) \\ &= \left(\sum_{a \in A_i} \sigma^*_i(a)u_i(a_i, \sigma^*_{-i})\right) - u_i(\sigma^*_i, \sigma^*_{-i}) \\ & = \sum_{a \in A_i} \sigma^*_i(a) (u_i(a_i, \sigma^*_{-i}) - u_i(\sigma^*_i, \sigma^*_{-i})) \\ & = \sum_{a \in A_i} \sigma^*_i(a) \text{Gain}_i(\sigma^*, a) && \text{ by the previous statements } \\ &= \sum_{a \in A_i} \left( C -1 \right) \sigma^*_i(a)^2 > 0 \end{align} where the last inequality follows since \sigma^*_i is a non-zero vector.
But this is a clear contradiction, so all the gains must indeed be zero. Therefore, \sigma^* is a Nash equilibrium for G as needed. \begin{align} &\mathbb{E}[\text{payoff for A playing H}] = (-1)q + (+1)(1 - q) = 1 - 2q, \\ &\mathbb{E}[\text{payoff for A playing T}] = (+1)q + (-1)(1 - q) = 2q - 1, \\ &\mathbb{E}[\text{payoff for A playing H}] = \mathbb{E}[\text{payoff for A playing T}] \implies 1 - 2q = 2q - 1 \implies q = \frac{1}{2}. \\ &\mathbb{E}[\text{payoff for B playing H}] = (+1)p + (-1)(1 - p) = 2p - 1, \\ &\mathbb{E}[\text{payoff for B playing T}] = (-1)p + (+1)(1 - p) = 1 - 2p, \\ &\mathbb{E}[\text{payoff for B playing H}] = \mathbb{E}[\text{payoff for B playing T}] \implies 2p - 1 = 1 - 2p \implies p = \frac{1}{2}. \end{align} Thus, a mixed-strategy Nash equilibrium in this game is for each player to randomly choose H or T with p = \frac{1}{2} and q = \frac{1}{2} . u_i(s^*) \geq \min_{s_{-i}} \max_{s_i} u_i(s_{-i}, s_i) . Every Nash equilibrium is minmax acceptable, as in a Nash equilibrium the following stronger property is satisfied: u_i(s^*) = \max_{s_i} u_i(s^*_{-i}, s_i) . But the opposite is not necessarily true.
For example, consider the following two-player game, where only pure strategies are allowed: One can check all 4 pure-strategy profiles (the 4 cells in the matrix), and verify that in all of them, one of the players has a profitable deviation, so there is no Nash equilibrium. On the other hand, the minimax value of the row player is 1 (R-T) and the minimax value of the column player is 1 (B-R). Hence, the strategy profile R-T is minmax-acceptable.
There are games in which even a minmax-acceptable strategy profile does not exist. For example, in Matching pennies, the minimax values (with pure strategies) are 1,1, but the sum of payments is 0, so no strategy profile can give 1 to both players. See also:
Original Nash papers
Nash, John (1950) "Equilibrium points in n-person games" Proceedings of the National Academy of Sciences 36(1):48-49. Nash, John (1951) "Non-Cooperative Games" The Annals of Mathematics 54(2):286-295.
Other references
Mehlmann, A. (2000) The Game's Afoot! Game Theory in Myth and Paradox, American Mathematical Society. Nasar, Sylvia (1998), A Beautiful Mind, Simon & Schuster. Aviad Rubinstein: "Hardness of Approximation Between P and NP", ACM, ISBN 978-1-947487-23-9 (May 2019), DOI: https://doi.org/10.1145/3241304 . # Explains the Nash Equilibrium is a hard problem in computation.
Text derived from https://en.wikipedia.org/w/index.php?oldid=1364609145 under CC BY-SA 4.0; modifications have been made.