Playing like an AI Poker Bot

Imperfect human play and strategies from state of the art algorithms

I looked down at KQo and sighed. After three hours of playing, I wasn’t feeling up for analyzing my opponent’s hands. I doubled my normal pre-flop raise because honestly, I wanted everyone to fold so I could collect their blinds. Not what I should have done with high cards, especially not when I was out of position.

“That’s a super strong bet under the gun,” a guy across from me quipped.

I had positioned myself as a nit girl so I could make these type of moves and get little wins when I got tired. One by one, each person raised their eyebrows and folded to my aggression … except one guy, who snap-called.

Playing poker at Wynn in Vegas

The flop came — ace and two low cards. Probably worst case scenario for me, as this guy very well could have an ace. He was first and I followed his check, when I should have bet to posture like I had an ace. The turn brought another unlinked low card, which couldn’t be good for either of us. I bet 10, trying to scare him off. I didn’t even try to value bet; I just wanted him to fold. He raised to thirty. I hemmed and hawed, then called him. That was a bad move — this guy didn’t bluff much based on the hands I’d seen him turn over throughout the night. For some reason, I had an intuition that he didn’t have top pair. Still, I cringed at my donation of twenty percent of my stack when an ace came on the river. I should have gone all-in, made some ballsy move to bluff him out. What did I bet? Two dollars. A measly percentage of the pot because I didn’t have the energy for bet sizing.  I had sat on my hands for an hour so I could pull off bluffs like this, and now I was weak? As the guy studied me for a few minutes, I kept my poker face even though I knew I should have bet more. Then, unexpectedly, he folded. 

“You had an ace, didn’t you?”

I smiled. “Guess you’ll never know.” 

“You would never bet that much pre-flop if you didn’t have an ace. I just had a middle pair.” At least my gut was right, if nothing else. 

I won the hand and a meaningful pot, even though I knowingly played like horribly. I wondered — what would have been optimal play in my situation, with the limited information I had? How would an AI have played differently? 

Currently the state of the art in poker AI is Pluribus — a 2019 bot that beat pros in six-player no-limit Hold ‘Em. As compared to the 2015 bot that solved heads-up poker, Pluribus deviates from a perfect counterfactual regret minimization algorithm. Pluribus uses a novel algorithm to search just a few moves ahead, rather than evaluate Nash equilibria for all possible game states. The bot also leverages faster self-play iterations to learn how to optimize hidden information games. While a layman won’t plug hands into an open-source version of Pluribus, there exist many accessible resources, such as podcasts from the algorithm’s creator, chatbots based on pros like AskFedor, and consumer-friendly solvers like PioSOLVER.

While we can glean surface-level learnings from these algorithms, AI poker bots largely remain black boxes. Plus, if I’m to learn anything from them, I also have to accept my short-comings as a human, not an AI. If I’m prone to forgetting where my keys are, I’m certainly not going to be able to play exactly like Pluribus. Especially when you take into account the fact that even just a heads up limit Hold ‘Em game tree has 316,000,000,000,000,000 branches. So rather than doing everything, I took away a few tangible tips from my studying of AI poker:

1. Don’t always use GTO-optimal bet sizing. Know the math, but mix it up with exploitative play. For example, AI poker bots have found that humans have difficulty processing abnormal betting behavior, such as way over or under betting.

2. Project a few moves ahead rather than considering all possible outcomes, especially since only ~30% of hands go to showdown.

3. If someone else raises pre-flop and you call, consider all hands you may have raised with in their position when you see the cards on the board. Calculate how to best play with this in mind. Apparently the bots made significant progress when they implemented this strategy.

4. Focus practice time online rather than in person. Due to speed of games, you’ll learn much faster — like a self-playing AI. Save the 12 hour days for the big tournaments, if that’s your goal.

I’m curious what new AI poker strategies will emerge now that the cultural zeitgeist has refocused on this technology. For now, I’ll adopt some of these strategies — playing riskier, narrowing my calculations. But I’ll also continue to be unlike bots — making dumb moves, playing in person more than I should for improvement’s sake. Because after all, what else is poker supposed to be than just pure fun.

---

Katie Mishra is a former mobile gaming founder who grew up writing books. Follow for musings and experiments while living in the arena.