Decision Theory and Nash Equilibrium in Poker

Game theory finds its way into everyday life and is always around us through many things we do, especially the decision theory branch. One more complex activity in which we can use game theory to our advantage is the great game of Poker. We shall see that poker, once you put on your game theorist glasses, is in theory a solvable game.

First off, let’s talk about poker. The legendary card game is actually a family of games, meaning that there are many ways and rules to play poker, however these variations don’t affect the objective of the game, which is to win other people’s money. We will focus on the “Texas hold-em” version of the game. In Texas hold-em, two players will post a mandatory bet, called Big Blind and Small Blind, before anything happens. This ensures that there is money on the table, basically providing something to play for. These people rotate throughout the game to make sure that everybody pays blinds fairly. After blinds are posted happens the Deal, players are dealt two cards each which are called hole cards. Then, players have the choice to push or fold. Pushing is betting money to stay in the game and folding is bailing out, leaving the money you already bet to the winner of the round. The next part is the Flop, where 3 community are dealt in the middle of the table. Each player can use these cards to create the best possible 5 card hand. After acknowledging the new cards, another round of betting occurs. Afterwards there is the Turn, which is when the fourth community card is dealt and another round of betting occurs. The same process then happens with a fifth community card, which is called the River. Finally, there is the Showdown, which is when players still in the game put their cards face up to see who wins the round. The winner takes the Pot, which is the entirety of the money bet on the table. Possible hilarity or extreme sentiment of guilt follows.

The play is clockwise and a player is on the Button every round. The player on the button acts before the small blind pre-flop while the big blind usually has the last chance to bet pre-flop. However, after the flop, the button has the last chance to act on each turns. Being on the button is a great advantage, since when the time comes to make your bet, other players have already revealed theirs and you have had the chance to gather the most information before taking action.  

There are four rounds of betting where players can get cues to what other people hold using psychology, probability and of course decision theory. Many kinds of moves can be done by a player who is up. The possibilities are not only to push or fold, you can also check, which is to pass, call, which is to match the current bet to stay in the game or finally to raise the bet.

Decision theory is applicable to Texas hold’em. As John Nash kindly proved, when mixed strategies are allowed, then a game with a finite number of players where each player can play with a finite number of strategies has at least one Nash equilibrium. As we know, a Nash equilibrium is an instance where "each player has chosen a strategy and no player can benefit by changing strategies while the other players keep theirs unchanged”. That specific set of strategy choices and the corresponding payoffs constitute a Nash equilibrium. This implies that in theory, every poker situation has a Nash equilibrium and that if every player played perfectly, even the most complex poker game would be a solved game.

However, there is a catch! It is one thing to know of the existence of a Nash equilibrium in a specific situation. It is a whole other thing to figure out what it actually is. Most situations will come to be highly complex and it will be quite a crispy challenge to find the Nash equilibrium in such circumstances.

Currently Nash equilibrium are easiest found in spots that are considered easy in Texas hold’em, such as a heads-up “pushbot” game, which is and all in/fold situation.  This is a very specific situation and a very small subset of the situations a poker player would potentially face.

It is important to understand that there are 1326 possible starting hands in a standard 52-card French deck. However, there are 2,598,960 possible 5 card combinations, which is eventually your hand when the community cards are revealed.

When evaluating a hand, players count outs. Outs are the cards still in the deck which could be combined with the player’s current hand in order to provide the player with a potentially winning hand. If a player has only one out in the deck, there is a 2% chance of that card turning up with 1 card to be drawn (so before the river), or 4% chance of that card turning up if 2 cards remain to be drawn (so before the turn). The rule of thumb is to multiply the number of outs by 2 or 4 depending of the number of cards to be dealt to get a rough approximation of the percentage or likelihood of your outs turning up. Another key point to keep in mind is called the range. Ranges are the set of possible hands an opponent could hold and take the same action with. For example, the shoving range of your opponent would be all the possible hands he would be comfortable going all in with.

In order to check is a given set of strategies are a Nash equilibrium is a heads up shove/fold game, we look at the following. From the SB’s perspective, given the hand range that BB is calling our shoves with, can we increase our expected value by folding any of the hands we are shoving or shoving any of the hands we are holding? From the BB’s perspective, given the hand range that the small blind is shoving, can our EV increase by calling any hands we are folding or folding any hands we are calling with?

If none of these strategies can be changed to increase expected value for each player, then we have a Nash equilibrium.

Now, let’s look at an example of a Nash equilibrium situation.

Suppose we are playing with 3 players and only the last 2 players standing make the money. One player has 1 chip and is on the button, we are the small blind and we have 2000 chips while the big blind has 3000 chips. Blinds are 120/240. The button folds and we are up. Now, our only option is to shove or fold. Nash equilibrium would be to shove ~13% and for the big blind to call ~7.5%. This means that in the long run with your current ranges, you would win in that situation approximately 13% of the time and the big blind with his current ranges would win around 7.5% of the time by calling your shove with his current range. Both these moves are very unprofitable, therefore the good strategy for us is to fold and let the BB win the blinds and go to the next round where the guy with one chip would probably bust out leaving us two on the table to take the money. Deviating from this strategy would not increase expected value for any players.

Here, we see an example of Nash equilibrium that doesn’t guarantee profit or even breaking even, which illustrates the important fact that playing with the Nash strategy will not make you a poker prophet who wins every hand.

In other situations, Nash equilibrium can also be found, but it is much more complicated. Calculators have been built to calculate ranges and probabilities of winning by shoving or folding in a heads up situation. Such wizardry can be found here: http://www.holdemresources.net/hr/sngs/hucalculator.html?action=calculate&r=10.0  

The best way to play poker with a mathematical strategy is to think about it as a function f(E, H) where E is the sequence of past events and H is the possible hands we might be dealt, returns a poker action such as (bet/call/check/fold). The example we saw had only to actions possible, push or fold, which shows a shortcoming of using strictly Nash strategy since it does not really apply to the real world. The more factors are included in E and the more actions are allowed to be taken, the more complex and accurate your function would be. 

If we were to try and figure out a complete strategy that incorporates previous rounds and take all possible hands, bluffs and learn new information on opponents as we go (like a self-learning algorithm), which would be to basically solve the game of poker, we would most likely be disappointed without results because apparently that wouldn’t even fit on all hard drives on this planet combined.

I hope this has shed a little more light on the game of poker, I look forward to discussing the subject in class with you guys.

Also, we will have more interesting stuff coming your way as Harry and I are working on this subject for our final project. Maybe we can take a field trip to the casino when we are done!

Cheers,

Sam

Sources:

http://blog.gtorangebuilder.com/2014/03/strategies-nash-equilibria-and-gto.html

http://www.icmpoker.com/en/blog/nash-calculator-and-nash-equilibrium-strategy-in-poker/

https://www.pokerstars.com/en/blog/team_pokerstars_blogs/ivan_demidov/2014/poker-and-the-nash-equilibrium-151970.shtml

http://www.chemical-ecology.net/java/possible.htm

http://www.flopturnriver.com/poker-dictionary/