Lecture 10: Taking Turns

For the past three lectures we’ve looked at simultaneous (or “static”) games, in which the players move without first observing the action of the other players. However, many games involve one player moving first, and then another player observing their move before choosing their response. Let’s see how introducing time dramatically influences the way we need to model games and solve for equilibrium.

Back to the penalty box

When we motivated the penalty-kicks game, we asserted that the goalkeeper didn’t have enough time to observe where the kicker kicked the ball: she had to guess and make her decision of where to dive without knowing the kicker’s choice.

But what if this weren’t the case? What if the ball was far enough that the keeper could see which direction it was going? In this case she could condition her decision on where the ball was kicked. In this case her strategy isn’t just “dive left” or “dive right” – rather, she has to specify what to do after each observed choice of the kicker.

To make this distinction, let’s introduce a bit of notation: let’s say that if the kicker kicks left $(\textcolor{3182bd}{L})$, the goalkeeper’s defensive actions may be labeled $\textcolor{e6550d}{L}$ and $\textcolor{e6550d}{R}$; but if the kicker kicks right $(\textcolor{3182bd}{R})$, the goalkeeper’s defensive actions may be labeled $\textcolor{e6550d}{L^\prime}$ and $\textcolor{e6550d}{R^\prime}$. Therefore the goalkeeper has four possible strategies:

• $\textcolor{e6550d}{LL^\prime}$: defend the left side of the goal no matter where the kicker kicks the ball
• $\textcolor{e6550d}{RR^\prime}$: defend the right side of the goal no matter where the kicker kicks the ball
• $\textcolor{e6550d}{LR^\prime}$: defend the same side of the goal where the kicker kicks the ball
• $\textcolor{e6550d}{RL^\prime}$: defend the opposite side of the goal where the kicker kicks the ball

Notice that for the goalkeeper, a strategy must contain instructions for what to do at each of the nodes: that is, as described above, her strategy space is not $\textcolor{e6550d}{\{L, R, L^\prime, R^\prime\}}$ but rather has to specify either $\textcolor{e6550d}{L}$ or $\textcolor{e6550d}{R}$ and $\textcolor{e6550d}{L^\prime}$ or $\textcolor{e6550d}{R^\prime}$. \(\textcolor{e6550d}{S_2 = \{LL', LR', RL', RR'\}}\) This brings us to the formal definition of a strategy, which is the most difficult definition for most students of game theory to internalize:

Definition (Strategy): A strategy is a complete, contingent plan for how a player will act at every decision node in a game.

The tricky thing here is that in order to be “complete and contingent,” a strategy must specify what a player does at every node in a game, regardless of whether that node is reached – even through their own actions!

The Extensive Form

It can be cumbersome and confusing to describe the timing of a game verbally, so economists generally illustrate games involving timing using the extensive-form representation of the game, which is a tree diagram illustrating the timing. A tree diagram consists of a set of nodes and edges. In a game tree there are four types of nodes:

• The initial node, indicating the beginning of the game.
• Decision nodes, at which a strategic player is making a decision. Each decision node indicates which player is making the choice.
• Chance nodes, at which a nonstrategic player (“nature” or “chance”) makes a random choice. For example, in some games a dealer deals a random card; because the dealer isn’t a strategic player, their move is modeled as a random event.
• Terminal nodes, when the game ends. Each terminal node is marked with the payoffs to each of the players.

The initial node is either a decision node (if the game begins with a player moving) or a chance node (if the game begins with a random event, like a dealer dealing cards).

In the penalty kicks game, the game starts when the kicker chooses whether to kick left or right; that is, the initial node is the one at the top of the diagram. The keeper observes what the kicker did, and makes a decision conditional on that. In the game tree, that means she has two decision nodes: one after observing the kicker chose $L$, and one after observing the kicker chose $R$. Once she makes her decision, the game ends and the payoffs are realized. Depending on the strategies chosen by the two players, a different terminal node is reached. Try clicking on the edges in the diagram to assign each player a strategy:

The requirement that a strategy describe a complete, contingent plan now has a visual meaning: a strategy must specify what happens at every decision node in the game. This will get tricky, as we’ll see in a few minutes! But before we move on, let’s talk about how the extensive form can still be used if there’s simultaneity.

Representing a Simultaneous Game using the Extensive Form

Note that we can actually use the extensive form to represent a simultaneous game. Think about the game of Rock, Paper, Scissors: simultaneity is key here. (Would you play a game if you had to go first, and the other player could choose the best option? Probably not.)

However, the highest-stakes game of Rock, Paper, Scissors (at least that I know of!) was not played in the normal way. Listen to this podcast if you have time; it’s awesome. Basically, an art collector had to choose between Christie’s and Sotheby’s to auction off his €20 million collection – and believing in luck, or fate, or just fun, he decided to have them compete for it by playing Rock, Paper, Scissors. But the way they did it was for each of them to submit their move in a sealed envelope to his lawyer. The envelopes would then be opened and the winner decided.

Let’s say Christie’s submits their (sealed) choice first, and Sotheby’s then chooses what to write down. Sotheby’s cannot make their choice contingent on what Christie’s has already done, because they don’t know what it was. So, their only choices are Rock, Paper, and Scissors. We can show this in a game tree by connecting the nodes after Christie’s move with a dotted line (sometimes this is shown by circling the nodes), which is called an information set.

Try clicking on the strategies for player 2, and notice that if you select a choice at one node, you’re selecting the same choice at all the nodes of the information set. In other words, “play Rock if they played Scissors” isn’t an option for player 2, because they don’t know what player 1 has done.

There is some esoteric debate about whether this is exactly the same game as simultaneous rock-paper-scissors…but as far as the strategy spaces go, it surely is. We’ll get much more into information sets in the next two weeks; for now, let’s think about how to solve a dynamic game.

Backwards Induction and the Need to Specify All Choices in a Tree

The definition of a strategy as a “complete, contingent plan” is one of the most confusing, in part because it doesn’t seem all that confusing. It’s a plan, right?

But look at the following devilish little game (sometimes called a “caterpillar game” since it kinda looks like a caterpillar). It shows an imaginary game show in which either one player gets €200, or they split a €600 pot. The timing of the game is as follows:

• First, player 1 chooses to Quit ($\textcolor{3182bd}{Q}$) or Play ($\textcolor{3182bd}{P}$). If they choose $\textcolor{3182bd}{Q}$, they get $\textcolor{3182bd}{€200}$, player 2 gets nothing, and the game ends. If they choose $\textcolor{3182bd}{P}$, the game moves to the second stage.
• In the second stage, player 2 has a chance to choose Quit ($\textcolor{e6550d}{Q}$) or Play ($\textcolor{e6550d}{P}$). Again, they can secure the $\textcolor{e6550d}{€200}$ by quitting; but if they choose to play, the game proceeds to the final stage.
• In the final stage, player 1 chooses to Take ($\textcolor{3182bd}{T}$) or Share ($\textcolor{3182bd}{S}$). If they’re greedy and choose $\textcolor{3182bd}{T}$, they get $\textcolor{3182bd}{€500}$ while player 2 gets only $\textcolor{e6550d}{€100}$. If they’re generous and choose $\textcolor{3182bd}{S}$, each of them gets €300.

Here’s the game tree. Try to select the strategies you think they’d play by clicking the edges of the tree:

It would be tempting to look at this and say that $\textcolor{3182bd}{Q}$ is a valid strategy for player 1: after all, if they choose that, the game ends and nothing else happens. Isn’t that a “complete, contingent plan” - just Quit right away, pocket the $\textcolor{3182bd}{€200}$ and walk away! But the counterintuitive aspect of this game is that player 1’s strategy must specify what they would do, if given the opportunity to take or share.

To see why, think about why player 1 would want to choose $\textcolor{3182bd}{Q}$. After all, if they choose $\textcolor{3182bd}{P}$, and player 2 also chooses $\textcolor{e6550d}{P}$, player 1 will walk away with either $\textcolor{3182bd}{€300}$ or $\textcolor{3182bd}{€500}$ – both of which are better than $\textcolor{3182bd}{€200}$! So, if player 1 chooses $\textcolor{3182bd}{Q}$ right away, it’s because they believe player 2 would choose $\textcolor{e6550d}{Q}$ if given the choice.

But why would player 2 do such a thing? Well, it’s because of what player 1 would do if player 2 chose Play. If player 1 would choose to share the pot ($\textcolor{3182bd}{S}$), then player 2 should choose $\textcolor{e6550d}{P}$ and get $\textcolor{e6550d}{€300}$ rather than $\textcolor{e6550d}{€200}$; but if player 1 would choose $\textcolor{3182bd}{T}$, then player 2 should choose $\textcolor{e6550d}{Q}$ and get $\textcolor{e6550d}{€200}$ rather than $\textcolor{e6550d}{€100}$.

To solve a game like this, therefore, we have to start from the end of each possible path, a process called backward induction. Here, the last decision node is player 1 choosing between $\textcolor{3182bd}{T}$, which gives them $\textcolor{3182bd}{€500}$, and $\textcolor{3182bd}{S}$, which gives them $\textcolor{3182bd}{€300}$. At that point they have a dominant strategy to choose $\textcolor{3182bd}{T}$; so choosing “$\textcolor{3182bd}{T}$ after $(\textcolor{3182bd}{P},\textcolor{e6550d}{P})$” has to be a part of player 1’s strategy. We can then work backwards: given that player 1 will choose $\textcolor{3182bd}{T}$ if they have the chance, player 2 will choose $\textcolor{e6550d}{Q}$; and knowing that player 2 will choose $\textcolor{e6550d}{Q}$, player 1 takes the money and runs at the beginning of the game.

Hence the backwards-induction solution to this game is the strategy profile $(\textcolor{3182bd}{QT},\textcolor{e6550d}{Q})$.

Subgame-Perfect Nash Equilibrium

Backward induction is a way to solve some games, but we have yet to have a formalized notion of equilibrium for dynamic games.

Recall that Nash equilibrium was a situation in which everyone is best responding to everyone else: that is, for each player $i$, their own strategy $s_i$ is a best response to the strategies being played by the other players, $s_{-i}$.

However, if we just keep that definition, we can land ourselves in trouble. Consider the following game:

The strategy space of player 1 is just whether they choose $\textcolor{3182bd}{X}$ or $\textcolor{3182bd}{Y}$: \(\color{3182bd}{S_1 = \{X,Y\} }\) For player 2, the strategy must specify what to do if player 1 chooses $\textcolor{3182bd}{X}$, and what to do if they choose $\textcolor{3182bd}{Y}$: \(\color{e6550d}{S_2 = \{AC,AD,BC,BD\} }\) The normal form of this game shows the payoffs for each strategy profile, and we can circle best responses like we did last lecture to see where the Nash equilibria are:

There are two Nash equilibria: $(\textcolor{3182bd}{X},\textcolor{e6550d}{BD})$ and $(\textcolor{3182bd}{Y},\textcolor{e6550d}{BC})$. But one of these doesn’t make sense! Look at what happens if player 1 chooses $\textcolor{3182bd}{Y}$: player 2 now has a choice between choosing $\textcolor{e6550d}{C}$ and getting a payoff of $\textcolor{e6550d}{1}$, or choosing $\textcolor{e6550d}{D}$ and getting a payoff of $\textcolor{e6550d}{0}$. If we were solving this using backwards induction, we would say that player 2 has to choose $\textcolor{e6550d}{C}$ at that point; indeed, the backward-induction solution says that $\textcolor{e6550d}{BC}$ is the only strategy player 2 should play. So, the Nash equilibrium $(\textcolor{3182bd}{X},\textcolor{e6550d}{BD})$ isn’t consistent with backwards induction.

The way game theorists “refined” the notion of Nash equilibrium was to say that a dynamic game (i.e. a game tree) consists of several subgames. We define a subgame as follows:

Definition (Subgame): A subgame consists of a decision node and all nodes reachable after that node, provided that none of the nodes in the subgame is connected to a node outside the subgame via an information set.

Given this, we can define a subgame perfect Nash equilibrium as:

Definition (Subgame perfect Nash Equilibrium): A strategy profile $s$ is a subgame perfect Nash Equilibrium if it is both a Nash equilibrium of the game as a whole and specifies strategies which are Nash equilibria of every subgame of the game.

It’s best to think of “subgame perfect” as an adjective describing a Nash equilibrium. In other words, we say that the Nash equilibrium $(\textcolor{3182bd}{X},\textcolor{e6550d}{BD})$ is not subgame perfect, while the Nash equilibrium $(\textcolor{3182bd}{Y},\textcolor{e6550d}{BC})$ is subgame perfect.

Note that subgames can be nested within one another. For example, in the caterpillar game, we could be interested in analyzing the subgame that begins after player 1 chooses $\textcolor{3182bd}{P}$, or the subgame that begins after both players choose $P$:

Note that these are called “proper subgames” because they are subsets of the game, and not the game itself. Technically, the entire game is a subgame of itself.

In the caterpillar game, you can see that the strategy profile $(\textcolor{3182bd}{QT},\textcolor{e6550d}{Q})$ is indeed a subgame perfect Nash equilibrium, because regardless of whether we look at the game as a whole or either of the proper subgames of that game, the strategies specified in that profile are equilibrium strategies in all of those subgames. Indeed, probably a better way of thinking about “backwards induction” is to think of it as “outwards induction” – that is, we start at the innermost subgame and work our way out.

Credibility

So far we’ve analyzed this from a pretty theoretical perspective; now that we’ve established our definitions, let’s see why the concept of subgame perfection is so critical in real life.

Fundamentally, subgame perfection is all about credibility. In particular, Nash equilibria that aren’t subgame perfect involve someone making a non-credible threat or promise.

Non-credible promises

Think again of the caterpillar game. Suppose player 1 said to player 2, “This is dumb. I’m just going to take 200 because you’re gonna take the 200 because you don’t believe I’ll split the 600 with you. I promise you, if you choose Play, I’ll choose Share. Deal?”

The issue with this is that when it comes time for player 1 to actually choose $T$ or $S$, they have a dominant strategy to take the money and run. So, the promise to play $S$ is a non-credible promise.

(Note, the twin assumptions that the payoffs represent all relevant payoffs, and that the game truly ends, are doing a lot of work here…next lecture we’re going to get to ongoing relationships, but for now we should accept the game at face value!)

Non-credible threats: entry deterrence

One of the classic examples of subgame perfection is a game describing entry deterrence. The story here is that there is a firm operating in an industry – say, the only fancy restaurant in a small town – and earning 400K per year. We call this firm the “incumbent.” A new chef (the “entrant”) is thinking about opening a second restaurant in this town, or another one with less demand for fancy food. If the entrant chooses the other town, the incumbent will continue to earn 400K, and the entrant will earn 100K. If they open a competing restaurant, though, the incumbent faces a choice: they can accommodate, meaning each restaurant will earn 200K; or they can engage in a price war, leaving them both with zero profits. The game tree here looks a little like an abbreviated version of the caterpillar game:

Again, if we just look at the normal-form representation of this game, there are two Nash equilibria:

The Nash equilibrium (Enter, Accommodate) is subgame perfect; once the new chef opens her restaurant, the incumbent shouldn’t fight. The strategy profile (Stay Out, Fight) is not subgame perfect, because it depends on the non-credible threat that the incumbent will Fight if the new chef chose to enter.

Continuous Strategy Spaces: From Cournot to Stackelberg

In this lecture we’ve focused on games with discrete strategies, but there are lots of games that take place over time in which players have a continuous strategy! Let’s wrap up by looking at how introducing timing to the Cournot game we’ve been analyzing affects the way the game is played.

We’ll keep the same framework: there are two firms who are each choosing a quantity $q_i$; the market price is $P(q_1,q_2) = 14 - q_1 - q_2$; each firm has a cost of 2 per unit. We showed that firm 2’s best response function to $q_1$ was \(\textcolor{e6550d}{BR_2}(\textcolor{3182bd}{q_1}) = \begin{cases} 6 - \tfrac{1}{2}\textcolor{3182bd}{q_1} & \text{ if }\textcolor{3182bd}{q_1 \le 12}\\ 0 & \text{ if }\textcolor{3182bd}{q_1 \ge 12} \end{cases}\) When we motivated the notion of Nash equilibrium, we said that this was a best response to what firm 2 believed firm 1 would do; but what if they could observe what firm 1 actually does? In other words, what if firm 1 moves first, and then firm 2 chooses what to do? In that case, the function above isn’t a best response: it is firm 2’s strategy!

Now, what is firm 1’s best response to this strategy? Well, because firm 1 knows that firm 2 will observe what it does, it knows exactly what firm 2 will do, as a function of its own choice of $q_1$. So its profit is no longer \(\pi_1(q_1,\textcolor{e6550d}{q_2}) = (14 - q_1 - \textcolor{e6550d}{q_2})q_1 - 2q_1\) but instead \(\pi_1(q_1,\textcolor{e6550d}{q_2(q_1)}) = (14 - q_1 - \textcolor{e6550d}{q_2(q_1)})q_1 - 2q_1\) Plugging in the function $q_2(q_1)$ we derived above into this gives us \(\pi_1(q_1) = \begin{cases} (14 - q_1 - \textcolor{e6550d}{\left[6 - \tfrac{1}{2}q_1\right]})q_1 - 2q_1 & \text{ if }q_1 \le 12\\ (14 - q_1- \textcolor{e6550d}{0})q_1 - 2q_1 & \text{ if } q_1 \ge 12 \end{cases}\) This is now just a function of $q_1$, so we can maximize it as we usually would! We can tell first of all that when the firm is optimizing, it must be that $q_1 < 12$, since $q_1 = 12$ would result in zero profit. So the relevant profit function is \(\begin{aligned} \pi_1(q_1) &= (14 - q_1 - \textcolor{e6550d}{\left[6 - \tfrac{1}{2}q_1\right]})q_1 - 2q_1\\ &= (8 - \tfrac{1}{2}q_1)q_1 - 2q_1\\ &= 6q_1 - \tfrac{1}{2}q_1^2 \end{aligned}\) Taking the derivative of this and setting it equal to zero gives us \(\pi_1^\prime(q_1) = 6 - q_1 = 0 \Rightarrow q_1^\star = 6\) Therefore the subgame perfect Nash equilibrium is \(\begin{aligned} \text{Strategy of firm 1: } & \textcolor{3182bd}{q_1 = 6}\\ \text{Strategy of firm 2: } & \textcolor{e6550d}{q_2(\textcolor{3182bd}{q_1}) = \begin{cases} 6 - \tfrac{1}{2}\textcolor{3182bd}{q_1} & \text{ if }\textcolor{3182bd}{q_1 \le 12}\\ 0 & \text{ if }\textcolor{3182bd}{q_1 \ge 12} \end{cases}} \end{aligned}\) This may be surprising! But note: firm 2’s strategy must be a complete, contingent plan of how to play the game. That means that regardless of what firm 1 does, firm 2’s strategy must specify what to do.

It might be a little easier to understand what’s going on if we think about a situation in which firm 1 has three strategies (produce $q_1 = 2$, 4, or 6), and firm 2 also has three strategies (produce $q_2 = 3$, 4, or 5). If the game is played simultaneously, and we draw out the payoff matrix, we see that the Nash equilibrium is at $q_1 = q_2 = 4$:

But if we look at a game tree where firm 1 chooses first, we can see that firm 1 will choose different quantities at each of its decision nodes:

In particular, if firm 2 chooses the best option at each of its three nodes, its strategy must be \(\text{Strategy of firm 2: }\textcolor{e6550d}{\boxed{q_2(\textcolor{3182bd}{q_1}) = \begin{cases}5 & \text{ if } \textcolor{3182bd}{q_1 = 2}\\4 & \text{ if } \textcolor{3182bd}{q_1 = 4}\\3 & \text{ if } \textcolor{3182bd}{q_1 = 6}\end{cases}}}\) and the best response for firm 1 to this strategy is \(\text{Strategy of firm 1: }\textcolor{3182bd}{\boxed{q_1 = 6}}\) which gives it a payoff of $\textcolor{3182bd}{18}$ (as opposed to $\textcolor{3182bd}{10}$ if it produces $\textcolor{3182bd}{2}$, or $\textcolor{3182bd}{16}$ if it produces $\textcolor{3182bd}{4}$). You can check that this is just a discrete version of the more general strategy function above.

Summary

• Games that take place over time are called dynamic games. They may be represented by a game tree; this is called the extensive form of the game.
• An information set connects nodes in a game tree at which the player doesn’t know which node has been reached. This can be useful in representing simultaneity in extensive-form games.
• A subgame of a game is a decision node and all nodes following it, provided that there are no information sets (so that the subgame is truly independent of the rest of the game).
• A Nash equilibrium is subgame perfect if it specifies strategies which are equilibrium strategies in every subgame of the game, including the game as a whole.
• One way of solving for subgame perfect Nash equilibria is to start at the end of the game and work backwards, a method called backwards induction.
• If players have continuous strategies, we represent their choices as functions of the choices made previously.