Lecture 9: Nash Equilibrium

The notion of equilibrium is core to economics, but it’s also one of the least well understood concepts.

Think about what equilibrium meant in the supply and demand context we looked at in weeks 2 and 3: we had individual agents best responding to prices by choosing quantities to buy or sell at those prices. Equilibrium occurred when the prices were such that the quantity supplied equaled the quantity demanded for each good. Once we were in equilibrium, no one had an incentive to behave in any way other than the way they were behaving, and there was no pressure on prices to move.

John Nash broadened and extended this notion of equilibrium to the kinds of games we have been looking at for the past two lectures. Whereas individual agents in the Edgeworth Box model responded to prices, individual players in a game respond to each other. In Nash’s telling, equilibrium then occurs when we have achieved a state in which each player’s strategy is a best response to the strategies chosen by the other players.

This article, written shortly after Nash’s tragic death in a car crash, offers a good introduction to Nash’s core ideas, why they were revolutionary, and what their limitations are. We’ll also talk about this at length in class, but for the purposes of this reading, I want to get to the simpler task of solving for Nash equilibrium in the kinds of games we’ve been analyzing. Let’s start with an easy example.

Games with a Unique Pure-Strategy Equilibrium

Formally, we can define a Nash equilibrium in pure strategies as follows:

Definition (Pure-Strategy Nash Equilibrium): A strategy profile $s = (s_1,s_2,...s_n)$ is a Nash equilibrium if, for every player $i$, $$u_i(s_i,s_{-i}) \ge u_i(s_i^\prime,s_{-i})$$ for all other strategies $s_i^\prime \in S_i$.

We can think of this definition in two ways – positively and negatively.

• Positively: if strategy profile $s$ is a Nash equilibrium, then every player is playing a best response to the strategies chosen by the other players.
• Negatively: if strategy profile $s$ is a Nash equilibrium, no player has a profitable unilateral deviation.

What does “profitable unilateral deviation” mean? Well, from each player’s perspective, what we’re doing is fixing what the other players are doing (i.e. $s_{-i}$) and asking if player $i$ is playing a best response, or if they could choose something which would make them better off – that is, a

• profitable (because it would make them better off)
• unilateral (because only they are changing their behavior)
• deviation (because it’s a different strategy than the one they would be playing in the equilibrium)

That’s awfully technical, but it really describes something quite simple. Let’s see what this looks like in a game with discrete strategies (i.e. a matrix game), and one with continuous strategies (Cournot duopoly).

Analyzing a matrix game: circling best responses

Toward the end of lecture 8, we looked at this game:

Last time we analyzed the game by applying the algorithm of iterated deletion of strictly dominated strategies, and we ended up with $(\textcolor{3182bd}{B},\textcolor{e6550d}{C})$ as the only strategy profile which survived that process.

Another way to analyze the game is to circle each player’s best response(s) to each of the other players’ strategies. For example, if player 2 were to choose $\textcolor{e6550d}{L}$, player 1’s best response would be to choose $\textcolor{3182bd}{M}$, since that gives a payoff of 4, which is higher than if she chose $\textcolor{3182bd}{T}$ (and got 1) or $\textcolor{3182bd}{B}$ (and got 3). So, we can circle the “$\textcolor{3182bd}{4}$” in the left-middle cell to indicate that this is a best response for player 1:

Try figuring out what player 1’s best response to $\textcolor{e6550d}{C}$ or $\textcolor{e6550d}{R}$ would be, and then also figuring out what player 2’s best response to $\textcolor{3182bd}{T}$, $\textcolor{3182bd}{M}$, or $\textcolor{3182bd}{B}$ would be; then check the boxes to make sure you’ve got it right.

You should find that when you’re done (or if you check all the boxes above) that the only cell with both payoffs circled is for the strategy profile $(\textcolor{3182bd}{B},\textcolor{e6550d}{C})$…just as we found when we did iterated deletion of strictly dominated strategies!

Think about what this cell means in the context of our positive and negative definitions of Nash equilibrium:

• Positive: for player 1, $\textcolor{3182bd}{B}$ is a best response to $\textcolor{e6550d}{C}$, and for player 2, $\textcolor{e6550d}{C}$ is a best response to $\textcolor{3182bd}{B}$. Therefore, every player is playing a best response to the strategy chosen by the other.
• Negative: Given that player 2 is choosing $\textcolor{e6550d}{C}$, player 1 has no profitable unilateral deviation: shifting from $\textcolor{3182bd}{B}$ to $\textcolor{3182bd}{T}$ or $\textcolor{3182bd}{M}$ would reduce their payoff from $\textcolor{3182bd}{2}$ to $\textcolor{3182bd}{1}$. Likewise, given that player 1 is choosing $\textcolor{3182bd}{B}$, player 2 has no profitable unilateral deviation: shifting from $\textcolor{e6550d}{C}$ to $\textcolor{e6550d}{L}$ or $\textcolor{e6550d}{R}$ would reduce their payoff from $\textcolor{e6550d}{1}$ to $\textcolor{e6550d}{0}$.

A lot of students look at this and quickly note that both players could do better: for example, by choosing to play ($\textcolor{3182bd}{M}$,$\textcolor{e6550d}{L}$). That’s true, but what would happen if they did? Then player 2 would have a positive unilateral deviation to $\textcolor{e6550d}{C}$, which would give them $\textcolor{e6550d}{4}$ rather than $\textcolor{e6550d}{3}$; and player 1 would respond to $\textcolor{e6550d}{C}$ with $\textcolor{3182bd}{B}$, and we’d find ourselves right back in equilibrium!

Plugging Best Response Functions Into Each Other: Cournot Equilibrium

With continuous strategies, as in the case of the Cournot model, we don’t have a matrix where we can circle best responses. But we do have best response functions, and we can use this same intuition: we want $\textcolor{3182bd}{q_1}$ to be a best response for player 1 to $\textcolor{e6550d}{q_2}$, and at the same time we need $\textcolor{e6550d}{q_2}$ to be a best response to $\textcolor{3182bd}{q_1}$ for player 2.

Recall from lecture 8 that our best-response functions were \(\textcolor{3182bd}{BR_1}(\textcolor{e6550d}{q_2}) = \begin{cases} 6 - \tfrac{1}{2}\textcolor{e6550d}{q_2} & \text{ if }\textcolor{e6550d}{q_2 \le 12}\\ 0 & \text{ if }\textcolor{e6550d}{q_2 \ge 12} \end{cases}\) \(\textcolor{e6550d}{BR_2}(\textcolor{3182bd}{q_1}) = \begin{cases} 6 - \tfrac{1}{2}\textcolor{3182bd}{q_1} & \text{ if }\textcolor{3182bd}{q_1 \le 12}\\ 0 & \text{ if }\textcolor{3182bd}{q_1 \ge 12} \end{cases}\) Let’s start by assuming that each firm produces less than 12 (which, it’s easy to see, will always be the case here…but it won’t always!). In that case, if firm 1 is best-responding to firm 2 and vice versa, we have \(\textcolor{3182bd}{q_1} = 6 - \tfrac{1}{2}\textcolor{e6550d}{q_2}\) \(\textcolor{e6550d}{q_2} = 6 - \tfrac{1}{2}\textcolor{3182bd}{q_1}\) This is just two equations in two unknowns, and we can plug one into the other to solve: \(\begin{aligned} \textcolor{3182bd}{q_1} &= 6 - \tfrac{1}{2}\underbrace{\left[6 - \tfrac{1}{2}\textcolor{3182bd}{q_1}\right]}_{\textcolor{e6550d}{BR_2}(\textcolor{3182bd}{q_1})}\\ \textcolor{3182bd}{q_1} &= 6 - 3 + \tfrac{1}{4}\textcolor{3182bd}{q_1}\\ \tfrac{3}{4}\textcolor{3182bd}{q_1} &= 3\\ \textcolor{3182bd}{q_1} &= \textcolor{3182bd}{4}\end{aligned}\) Plugging this back into the best-response function for $\textcolor{e6550d}{q_2}$ gives us \(\textcolor{e6550d}{q_2} = 6 - \tfrac{1}{2} \times \textcolor{3182bd}{4} = \textcolor{e6550d}{4}\) So, the Nash equilibrium of this game is $(\textcolor{3182bd}{4},\textcolor{e6550d}{4})$. (Note that these are the strategies themselves, not the payoffs: that is, the Nash equilibrium is that each firm produces 4 units of output.) Visually, we can see that this is where the best-response curves intersect:

Note that one feature of the Cournot equilibrium is that it is stable in a very meaningful sense. In particular, suppose we assume that this game is played each day, and that each firm “best responds” to the action taken by the other firm the day before. Suppose one day, firm 1 chose $q_1 = 2$. Firm 2 would respond by producing $q_2 = 6 - \tfrac{1}{2} \times 2 = 5$. Firm 1 would then respond to that the following day by producing $q_1 = 6 - \tfrac{1}{2} \times 5 = 3.5$. The best response to 3.5 is 4.25, the best response to 4.25 is 3.875, and as the process goes on the quantities converge to 4. This is a little like the story of how excess supply and excess demand drive prices to the equilibrium price in the competitive model we looked at in Econ 50.

One last thing to note before we move on: there is a special class of games called symmetric games, of which this specific instance of the Cournot setup is an example. In a symmetric game, the strategies and payoffs available to each player are identical. So in this case, each firm is choosing $q_i$, and its payoff function is \(\pi_i(q_i,q_{-i}) = (14 - q_i - q_{-i})q_i - 2q_i\) In such a case, the best-response functions will also be identical: that is, we had \(BR_i(q_{-i}) = 6 - \tfrac{1}{2}q_{-i}\) If this is the case, we are looking for a strategy which is a best response to itself. If we label this $\hat q$, we have \(\hat q = 6 - \tfrac{1}{2} \hat q\) Solving gives us $\hat q = 4$; that is, the equilibrium is that both firms produce $\hat q = 4$.

Not all games are symmetric, of course! In the homework you’ll look at a Cournot model in which firms have different costs; therefore they have different best-response functions, and produce different amounts in equilibrium.

Games with Multiple Equilibria

In the two games we looked at above, there was a single Nash equilibrium. But one of the most salient features of these kinds of equilibria is that there can be multiple equilibria!

We’ll talk at length in lecture about each of these games, but for the purposes of the reading let’s just introduce them and see why they have multiple equilibria.

Coordination Games

One broad class of games with multiple equilibria are called coordination games. In a coordination game, all players get a higher payoff if they can coordinate their actions with each other. Think of a bike path on campus. Suppose you and another student are approaching one another on your bikes, traveling in opposite directions. Each of you has to choose whether to pull to the left or the right. If you each go to your own left, you’re fine; likewise if you each go to your right. But if you go to your left and they go to their right (or vice versa) you get into a crash. If we assign a payoff of 1 to not crashing, and 0 to crashing, then we can model this game as

Note that both $(L,L)$ and $(R,R)$ are perfectly fine equilibria; it doesn’t matter if you both choose left or right, as long as you are able to coordinate your decisions.

In a game like this it can be useful to have a third party “impose” an equilibrium: require by law, for example, that everyone drive on the right side of the road. Note, though, that having a law is neither necessary nor sufficient for equilibrium. For example, a simple coordination mechanism is a traffic light: it informs all cars of when it is their turn to go, and thereby (ostensibly) avoids collisions. But in one of my favorite speeches ever by an economist, Avinash Dixit’s Presidential Address to the American Economic Association, he showed two videos. The first was this one from St. Petersburg, in which a hapless traffic light attempts without success to coordinate the behavior of drivers:

The second was this one of an intersection in India, where drivers (and bikers, and pedestrians) managed to coordinate with one another without a traffic light at all.

In other words, a coordination mechanism can “select” a particular equilibrium of the game; but it only stays in equilibrium if everyone dutifully follows its instructions.

Pareto Coordination Games and Rousseau’s Stag Hunt

In the above game, everyone viewed the two equilibria equally: it didn’t matter if you both chose left or both chose right, as long as you agreed.

But some games have multiple equilibria in which one is better for everyone than the other one. For example, suppose two fraternities are throwing parties, and you and your friend are each planning on going to one of them. Unfortunately, you spent all Econ 51 lecture playing Clash Royale in the back row, and your phone is out of battery; so you can’t coordinate with your friend. You have to show up to one or the other, and hope they’re there; they have to do the same.

If the two frats were throwing equally good parties, we’d be back in the pure coordination game above. But suppose Frat A has better music and altogether better vibes than Frat B. So, both you and your friend would get a payoff of 2 if you both show up to the party at frat A, as opposed to both getting a payoff of 1 if you both choose B:

Does this make sense? Perhaps surprisingly, situations like this are not only possible, but common! There are lots of situations in which there are multiple equilibria, one equilibrium would be best for everyone, but because everyone is “stuck” in a bad equilibrium, there’s no way to shift to the good one.

One famous example of this is the “stag hunt” parable from Jean-Jacques Rousseau’s Discourse on Inequality. In it, he tells the story of two hunters who go into the woods to hunt a stag. If they catch the stag, they each get a payoff of 5. However, they can only catch the stag if they work together. Each of them can break off from the hunt and catch a hare (a kind of large rabbit) for sure; anyone who chooses to hunt the hare gets a payoff of 4. Therefore, the payoffs are as follows:

Again there are two Nash equilibria. Which would you do in this situation? We’ll find out in class.

Games of Conflict

In the above game, everyone at least agreed on which was the better option. But what if there was disagreement?

Let’s think about the frat party example again, but now suppose you and your friend have different taste in music: you like the music at frat A better, and they like the music at frat B better. You’d still rather go to the same party (and get a payoff of 0 if you go alone to either one), but now you have a higher payoff if you both choose A, and they have a higher payoff if you both choose B:

Let’s dial up the conflict one more notch. Suppose you and a rival are both applying for internships at two companies, Accenture and Bain. The Accenture internship pays 8K; the Bain job pays 10K. If you don’t get a job, you have a payoff of 0. If you apply for the same job, you each have a 50% chance of getting it – in other words, if you both apply to Accenture, you each have an expected payoff of 4K, and if you both apply to Bain, you each have an expected payoff of 5K. If you apply for different jobs, you are guaranteed to get the job, so if you choose Accenture and your rival chooses Bain, you get 8K and they get 10K.

Therefore the payoffs, in expected value of thousands of dollars, are given by the following table:

Again, we’ll play through this in class: but the key point is that there are two equally plausible equilibria, one of which you prefer, and the other of which is better for your rival.

Equilibria in Mixed Strategies

Up to now we’ve looked at equilibria in pure strategies; that is, each player is choosing one of their strategies and playing it with 100% certainty, and each is choosing a best response to the other.

Some games don’t have equilibria in pure strategies, though! Recall the penalty kicks game from lecture 8. If we circle the best responses, we can see that there is no cell in which both payoff are circled:

This makes sense: if the kicker kicks to the left, the goalkeeper wants to defend left; but if the goalkeeper is defending left, the kicker would want to go to the right! This type of game is called a zero-sum game: there’s going to be a winner and a loser, so you’ll never have the two coordinating with one another in equilibrium.

There is a Nash equilibrium to the game, however. Recall that a player can play a mixed strategy: that is, play each of their strategies with some probability that isn’t 1. Let’s say in particular that the kicker kicks to the left with probability $\textcolor{3182bd}{p}$, and the goalkeeper defends to the left with probability $\textcolor{e6550d}{q}$. That is, the kicker (player 1) is playing the mixed strategy $\textcolor{3182bd}{\sigma_1 = (p,1-p)}$ and the goalkeeper (player 2) is playing the mixed strategy $\textcolor{e6550d}{\sigma_2 = (q, 1-q)}$. What are each of their best responses to the other’s strategy?

Let’s think about it from the kicker’s perspective. We found in Lecture 8 that if they believed that the keeper would defend left with probability $q$, their payoff to kicking left or right was \(\begin{aligned} \textcolor{3182bd}{u_1}(\textcolor{3182bd}{L}, \textcolor{e6550d}{q}) &= \textcolor{3182bd}{70}\textcolor{e6550d}{q} + \textcolor{3182bd}{90}\textcolor{e6550d}{(1-q)} = \textcolor{3182bd}{90-20q}\\ \textcolor{3182bd}{u_1}(\textcolor{3182bd}{R}, \textcolor{e6550d}{q}) &= \textcolor{3182bd}{80}\textcolor{e6550d}{q} + \textcolor{3182bd}{50}\textcolor{e6550d}{(1-q)} = \textcolor{3182bd}{50+30q}\\ \end{aligned}\) These were equal when $q = 0.8$; so the kicker’s best response, as a function of the belief $q$, was: \(\textcolor{3182bd}{BR_1}(\textcolor{e6550d}{q}) = \begin{cases} \{\textcolor{3182bd}{L}\} & \text{ if }\textcolor{e6550d}{q < 0.8}\\ \{\textcolor{3182bd}{L},\textcolor{3182bd}{R}\} & \text{ if }\textcolor{e6550d}{q = 0.8}\\ \{\textcolor{3182bd}{R}\} & \text{ if }\textcolor{e6550d}{q > 0.8} \end{cases}\) Now, if we think about $\textcolor{3182bd}{p}$ as the probability that the kicker kicks left, we can analyze this as follows: if the keeper is defending to the left less than 80% of the time ($\textcolor{e6550d}{q < 0.8}$), the kicker should kick left ($\textcolor{3182bd}{p = 1}$). If the keeper is defending to the left more than 80% of the time ($\textcolor{e6550d}{q > 0.8}$), the kicker should kick right ($\textcolor{3182bd}{p = 0}$).

But, if the goalkeeper is defending to the left exactly 80% of the time ($\textcolor{e6550d}{q = 0.8}$), kicking to the left or right yields the same expected payoff, so the kicker could play any mixed strategy $\textcolor{3182bd}{(p,1-p)}$ with $\textcolor{3182bd}{p} \in [0,1]$ and get the same outcome.

This is important enough that it’s worth capturing in a lemma:

Lemma : If multiple pure strategies are best responses to what other players are doing, then a mixed strategy $\sigma_i$ that assigns positive probability to those strategies (and no others) is also a best response.

In other words, when $q = 0.8$, both $\textcolor{3182bd}{L}$ and $\textcolor{3182bd}{R}$ are best responses, so the mixed strategy “play $\textcolor{3182bd}{L}$ with probability $\textcolor{3182bd}{p}$ and $\textcolor{3182bd}{R}$ with probability $\textcolor{3182bd}{1-p}$” is also a best response, for any $\textcolor{3182bd}{p} \in [0,1]$.

Since $p = 1$ corresponds to strategy $L$ (kick left all the time) and $p = 0$ corresponds to strategy $R$, we can therefore write the kicker’s best response as their own probability $p$, as a function of the goalkeeper’s probability $q$: \(\textcolor{3182bd}{p = BR_1(\textcolor{e6550d}{q})} = \begin{cases} \textcolor{3182bd}{1} & \text{ if }\textcolor{e6550d}{q < 0.8}\\ \textcolor{3182bd}{[0,1]} & \text{ if }\textcolor{e6550d}{q = 0.8}\\ \textcolor{3182bd}{0} & \text{ if }\textcolor{e6550d}{q > 0.8} \end{cases}\) This is a little like the Cournot model, where we had a continuous strategy space, only now the continuous strategy space is the probability with which you kick left. And in fact we can draw the best-response curve just as we did before:

If we do the same exercise with the goalkeeper, we can find that the keeper will defend to the left if they think there’s at least a 60% chance the kicker will kick there: \(\textcolor{e6550d}{q = BR_2(\textcolor{3182bd}{p})} = \begin{cases} \textcolor{e6550d}{1} & \text{ if }\textcolor{3182bd}{p > 0.6}\\ \textcolor{e6550d}{[0,1]} & \text{ if }\textcolor{3182bd}{p = 0.6}\\ \textcolor{e6550d}{0} & \text{ if }\textcolor{3182bd}{p < 0.6} \end{cases}\)

The only pair of strategies which are best responses to one another, therefore are: \(\textcolor{3182bd}{\sigma_1 = (0.6, 0.4)}\) \(\textcolor{e6550d}{\sigma_2 = (0.8, 0.2)}\) Intuitively: if the kicker is kicking to the left 60% of the time, the goalkeeper is indifferent between defending left and right, so every mixed strategy – including defending to the left 80% of the time – yields the same expected payoff. So, $\textcolor{e6550d}{\sigma_2 = (0.8,0.2)}$ is a best response to $\textcolor{3182bd}{\sigma_1 = (0.6,0.4)}$. Likewise, if the goalkeeper is defending to the left 80% of the time, the kicker is indifferent between kicking left and right, so every mixed strategy – including kicking left 60% of the time – yields the same expected payoff. Therefore, $\textcolor{3182bd}{\sigma_1 = (0.6,0.4)}$ is a best response to $\textcolor{e6550d}{\sigma_2 = (0.8,0.2)}$ as well. Since each of these mixed strategies is a best response to the other, they constitute a Nash equilibrium!

Importantly, note that the kicker isn’t trying to make the goalkeeper indifferent, or vice versa. The kicker would love to convince the keeper that she’s going to go left 100% of the time, and then kick to the right! What we’re saying here is that the only equilibrium is one in which the kicker kicks to the left 60% of the time, and the defender defends left 80% of the time.

One last note: if a player has more than two strategies available to them, a mixed strategy need not put positive probability on all of them; rather, just those in the best response set. For example, we looked at the game in which the kicker had a third option of kicking down the middle:

If the goalkeeper is playing $\textcolor{e6550d}{\sigma_2 = (0.8, 0.2)}$, the payoff for the kicker to kicking left or right is 74, while the payoff to kicking down the middle is 72. The mixed strategy she plays in equilrium is therefore \(\textcolor{3182bd}{\sigma_1 = (\underbrace{0.6}_L, \underbrace{0}_M, \underbrace{0.4}_R)}\) One way to find the mixed-strategy equilibrium (or any Nash equilibrium!) is therefore to first apply the process of iterated deletion of dominated strategies, and then analyze the reduced game.

Let’s finish up by looking at the situation in which there are multiple equilibria in both pure and mixed strategies.

Games with Both Pure and Mixed Strategies, and an Odd Result

For our last game, let’s go back to the internship problem. We found that there were two equilibria in pure strategies: one of you applies to Accenture, and the other to Bain. However, there is also a mixed-strategy equilibrium.

Because this is a symmetric game, we can consider just one probability: the probability with which each of you applies to Accenture. Let’s call this $p$. If the other player is applying to Accenture with probability $p$ and Bain with probability $1 - p$, the expected payoffs from applying to each is \(u_i(A,p) = 4p + 8(1-p) = 8 - 4p\) \(u_i(B,p) = 10p + 5(1-p) = 5 + 5p\) These are equal when \(\begin{aligned} 8 - 4p &= 5 + 5p\\ 3 &= 9p\\ p &= \tfrac{1}{3} \end{aligned}\) So if each of you is applying to Accenture with probability $p = {1 \over 3}$, and Bain with probability $1 - p = {2 \over 3}$, then each of you is indifferent and therefore willing to play a mixed strategy, including $p = {1 \over 3}$!

If we plot the best response functions, we can see all three Nash equilibria:

The two pure-strategy Nash equilibria occur at $(\textcolor{3182bd}{0},\textcolor{e6550d}{1})$ and $(\textcolor{3182bd}{1},\textcolor{e6550d}{0})$: if your rival is definitely choosing Accenture, you should definitely choose Bain, and vice versa. The mixed-strategy Nash equilibrium occurs at $(\textcolor{3182bd}{1 \over 3},\textcolor{e6550d}{1 \over 3})$.

Note that in the penalty kicks game, there was a single Nash equilibrium; here there are two equilibria in pure strategies, and one in mixed strategies. In fact, Stanford economist and Nobel Prize winner Robert Wilson proved in 1971 that “almost all” finite games have an odd number of Nash equilibria! It’s a fun result that we’ll go into in more detail in Q Club this week, so check that out if you’re interested.

Summary

• Nash equilibrium exists when each player is playing a best response to all the other players
• For a game with discrete strategies that can be represented by a payoff matrix, we can find the pure-strategy Nash equilibria by circling best responses and finding cells in which both players’ payoffs are circled. For a game with continuous strategies, we find the Nash equilibria by plugging one player’s best response function into the other.
• A player is only willing to mix between strategies when they are indifferent between those strategies. Therefore, if a player is playing a mixed strategy in a Nash equilibrium, the other players must be playing a strategy such that the player is indifferent between the strategies over which they are mixing.
• Sometimes there is a single Nash equilibrium in pure strategies; sometimes there are multiple; sometimes there are none, and the only possible NE is one that involves mixed strategies!