How computers were finally able to best poker pros
By Devin Powell
Feb 4 2017
Twelve days into the strangest poker tournament of their lives, Jason Les and his companions returned to their hotel, browbeaten and exhausted. Huddled over a pile of tacos, they strategized, as they had done every night. With about 60,000 hands played — and 60,000 to go — they were losing badly to an unusual opponent: a computer program called Libratus, which was up nearly $800,000 in chips.
That wasn’t supposed to happen. In 2015, Les and a crew of poker pros had beaten a similar computer program, winning about $700,000. This time, the pros had initially kept things more or less even by finding flaws in how the computer played; fans following this “Brains Vs. AI” competition at the Rivers Casino here put the odds of the AI winning at only about 1 in 4.
But by the second week, the flaws had disappeared; the odds of the computer triumphing rose. “On Day 1, it had played well, but it wasn’t impressive,” Les said. “What’s impressive is how this thing has learned and evolved, how much better it has gotten every day.”
Machines have learned a lot about how to play games. Twenty years ago, they figured out checkers, and 10 years ago they toppled the Russian grandmasters of chess. Even China’s game of go has been solved. But poker remained firmly in the hands of humans.
That’s because unlike checkers and chess, where all the pieces are visible, poker is a game of limited knowledge and uncertainty, of hidden cards and bluffs. It is perhaps truer to life, which may explain why it has been difficult for silicon chips to grasp.
“AIs have had a lot of trouble with poker,” said Noam Brown, a graduate student at Carnegie Mellon University who developed Libratus with CMU computer scientist Tuomas Sandholm. “It’s the holy grail of imperfect information games.”
A victory for Libratus, Brown said, would not be much of a threat to human poker players. Its brain is a supercomputer that costs millions of dollars per year to run, so using it to play poker would not be a great way to make money. But Libratus could be a step toward helping artificial intelligence deal more broadly with uncertainty.
That’s because poker is not simply a game of chance. Neither does it require being able to read an opponent’s facial expressions, although Hollywood might like us to believe otherwise. What guides Libratus’s decisions is powerful mathematics, math that could be applied to auctions, negotiations, finance, security and other real-world arenas in which information is hidden.
Serious mathematicians have long been fascinated by poker. John von Neumann, a pioneer in game theory, the branch of mathematics that deals with competition, explored the ins and outs of the card game early in the past century. So did John Nash, whose struggle with schizophrenia was depicted in the movie “A Beautiful Mind.” In 1950, Nash published a paper showing that there is a best strategy for many games, including one-on-one poker, regardless of how your opponent plays. That strategy, now called a Nash equilibrium, may not always win, but it does better than any other approach.