Yale and the Beast: Adventures in Game Theory Part II
What a Yale course and a reality TV show can teach us about the widely understood "mixed strategy."
In the first part of Adventures in Game Theory, I wrote about the Yale Open Courses, the 2/3 the Average game, and a challenge in the controversial Beast Games TV show.
In this second installment, we’ll discuss another topic that is very dear to me and games players everywhere: the misunderstood Mixed Strategy. A mixed strategy sounds simple at first. It just means a mix of different strategies are employed in order to maximize your win rate or avoid getting exploited. For example, your strategy in Rock-Paper-Scissors will be a mixed strategy.
In poker too, mixed strategies are extremely common when it comes to the theory of the game—-it means doing different things with the same hand. In my last post, I wrote about infinite iterations driving down the Solution of the 2/3 the Average Game to 0. Mixed strategies are also the result of iterative play, and the idea that you don’t play one poker hand in a vacuum, but that it exists in a multiverse.
But in the real world, the right way to play a poker hand is usually a lot simpler and doesn’t require words like “iterations” or “multiverse.” Enter Professor Ben Polak.
He spends about half of his class on Mixed Strategies explaining that a Pure Strategy is a type of Mixed Strategy.
The length of his explanation on this starts to get funny. For example, he spends about five minutes explaining that if there’s a game where you have to pick the tallest guy, and there are three guys, who are 6 feet, 5 feet 10 and 5 feet 8, you should adopt a “pure strategy” of picking the tallest guy each time.
Wait, what? He’s explaining that the tallest guy is the tallest guy.
He does the same thing with students and their GPAs.
At around this point, I couldn’t help but wonder: Why is he doing this?
It’s because he knows that once curious students dive into mixed strategies, they may get seduced into seeing them everywhere. Students fall into a rabbit hole of using them even when they don’t apply. Poker players reading this may feel a shiver of recognition.
I can just picture Professor Polak talking about “mixed strategies” and poker in the exact same way.
“If the game is to pick the tallest gal, should I pick the gal who’s five feet eight, or the one who’s five feet four?”
“In poker, should I call this river when I know my opponent is bluffing too much, or should I fold?”
Both questions seem absurd right? That’s because they are the same question. Some wise words from Polak to encapsulate the lesson: “Everything in which I am mixing is itself best. And the reason is if it wasn’t, kick out the thing that isn’t best.”
Step Your Way to the Win
Once we understand that a pure strategy is a type of mixed strategy, it’s easier to see where mixed strategies really do apply. Let’s return to an episode of the Beast Games to see a case in point.
We are down to three contestants playing to win the deed to a private island. At this point, Mr. Beast introduces the 5-3-1 game. Each round, the three contestants secretly pick one of three possible numbers: 5, 3 or 1. The twist is that if multiple players pick the same number, neither get to move at all. The goal is to advance as many steps as possible. So in the screenshot above, each player steps ahead 1, 3 and 5 steps respectively.
So how to approach the game? If we all pick five all the time, the game is frozen until one player exploits this and starts playing three. But then, another player may realize this and start playing three also. But then the third player wins by sticking with “five.” As we iterate, it becomes clear: a mixed strategy must be the solution.
The basic solution, without accounting for things like approaching the end of the “race”: We pick 3 about 44% of the time and 5 about 56% of the time. If you want to understand why the numbers seem surprisingly close to each other, check the footnotes1.
The Beast version of the game wasn’t well executed because there was a “twist” that ruined the game: the player who wins the race gets to pick a second player to advance with them2.
This did get me thinking more about the optimal strategy for the 5-3-1 game in a winner take all scenario.
To 1 or Not to 1
The first question that perplexed me was “do we ever pick one to start the game?” After doing some math on the back of an envelope, it seemed like we should choose 1 a small percentage of the time. Until a friend3 reminded me: The power of picking a larger number is not just that you may get to step ahead 3 or 5 steps. Even if you’re frozen for that round, you probably “blocked” one of your opponents from moving 3 or 5 steps ahead.
But I can see why I made the mistake. If we change the parameters, it may be reasonable to play “1” once in a while. For example, what if the game isn’t zero sum? Suppose that each player wins $100 per point, and the game lasts a fixed number of rounds. In that case we may want to include “1” into the mix. Notice the difference? One version is about maximizing our value. In the other game, we are trying to out pace our opponents to become the sole winner.
When I hear someone talk about an “abundance” mindset, it reminds me of this distinction. Sometimes we are competing in a world of abundance. Like a spin class: we can all get fitter, and if I get my highest score, it doesn’t matter if your score is higher than mine. We both won the more important game of abundance in fitness! But there are other situations, like a job application for a specific position, which are much closer to winner take all competition.
The Lesson: If you want to come up with a solution, you better be aware of the game’s payoffs. Initially I framed things incorrectly. I forgot that we need to think defensively as well as offensively in this Winner Take All game.4 In life it’s often healthier and more profitable to have an abundance mindset, as it attracts people and opportunities. But in competitive games, you need to be cut-throat.
Mixing up 5 and 3
So let’s assume that the game is a race not a spin class. We can totally eliminate playing “1.” So we decide to play our mix of five and three — it seems like our opponents have devised similar strategies.
Great, we have a fair fight. Sometimes we’ll all pick 5 or 3 and no one will step. In other combinations, one of us will get lucky and choose 3 (or 5) while the other two players chose 5 (or 3).
Imagine that you are a player in this game, up against two fictional opponents, Matt and Jessica. But wait a second. Matt seems to have a timing tell on whether he’s going to play 5 or 3—he is more straightforward when he writes down 5, and pauses before writing down 3. Detecting this tell, you are able to win more often by picking the inverse of the number he chose. If he chooses 5, you choose 3 and vice-a-versa. Forget that mix you calculated. Now you’re largely reacting to Matt’s tell. Regardless of what Jessica picks, you or Matt will advance.
In most games, spotting a tell means shifting equity from your opponent to you. That seems fair enough right? But in this 5-3-1 game, that’s not the case. You win equity, and so does Matt.
Who loses? Jessica!
Wait? But no one cheated. Jessica just lost because she wasn’t as bad as Matt? How is that fair? It’s not.
The lesson: Solutions in three-handed games are so very fragile! Don’t try and fit a square into a triangle hole. And if you’re Jessica, wake up and pay attention, so you don’t end up being the unlucky loser.
The top ranking tournament poker pro of 2023, Ike Haxton, was recently on a podcast, where he said a lot of assumed best strategies for three-handed play may be wrong. As he says, “It was a surprise when people took counter factual regret minimization — the algorithm that underpins every solver we use—and ran it with more two players, and it spouted out something that wasn’t complete nonsense.” [video timestamped to 20:11]
Three player dynamics are great fun to analyze in part because there is so much to discover. Indeed, Haxton predicts there may be “a leap forward” in our future understanding of multiplayer poker.
Whether by steps or by leaps, so much of gameplay is not just in knowledge, but in priorities, logic and staying present. As Polak reminds us so beautifully, the best move right now is the best move.
Related links:
Adventures in Game Theory Part I: Yale and the Beast
The ratio we want is 5:3 so my immediate thought was to choose three 3/8 of the time, and 5 5/8. Which would be 38% and 62%. But that’s not the right way to set up the equation in a three-player game. We need to square the numbers to get the right equation. That’s because we are trying to “unmatch” our opponents, and so if Player A and B uses these basic probabilities, they’ll overplay “5” and allow the third opponent to advance quickly by playing 3 more often. For example, .62*.62= .38 while .38*.38=.14. Once you multiply the probabilities, it’s much more intuitive.
The 5-3-1 game seems like an interesting game to turn three players into one winner. But it’s a poor game to turn three players into two winners, especially since talking was permitted in the Beast Games version so the players who wanted each other to advance simply talked to each other. (and this was NOT against the rules.)
My friend: Bill Chen, co-author of the Mathematics of Poker
It follows that we should give 5 a greater weight than 55.6% because of the extra “blocking effect” of five. The change should be subtle because of the nature of three-player equilibriums and geometric growth. If we adjust too dramatically, we allow our opponents to exploit us by playing 3. The case of “1” is much different because it’s such a low number and it unblocks both 5 and 3.
Wonderful exposition on game theory. If I remember correctly, years ago on the game show Jeopardy there were notable instances of players betting incorrectly in Final Jeopardy. I would guess that game theory is now sufficiently well-known that that doesn't happen much anymore.
I love these game theory posts!