Poker Is the Latest Game to Fold Against Artificial Intelligence
In a landmark achievement for artificial intelligence, a poker-bot developed by researchers in Canada and the Czech Republic has defeated several professional players at heads-up no-limit Texas hold’em poker.
Recommended for You
Recommended for You
Perhaps most interestingly, the academics behind the work say their program overcame its human opponents by using an approximation approach that they compare to “gut feeling.”
“If correct, this is indeed a significant advance in game-playing AI,” says Michael Wellman, a professor at the University of Michigan who specializes in game theory and AI. “First, it achieves a major milestone (beating poker professionals) in a game of prominent interest. Second, it brings together several novel ideas, which together support an exciting approach for imperfect-information games.”
Later this week, a tournament at a Pittsburgh casino will see several world-class poker players play the same version of poker against a program developed at CMU. Tuomas Sandholm, a professor of computer science at CMU who is leading the effort, says the human players involved are considerably stronger than those tested by the Alberta researchers, and 120,000 hands will be played over 20 days, providing greater statistical significance to the results. The tournament could confirm that AI has indeed mastered a game that has long seemed far too complex and subtle for computers.
DeepStack, the poker-playing software that has already bested some professional players, was developed by a team led by Michael Bowling, a professor of computer science at the University of Alberta, which included researchers from Charles University and Czech Technical University in the Czech Republic. In a research paper posted online but not yet peer reviewed, the researchers say that DeepStack played almost 45,000 hands of poker against several players, beating them handily.
Poker differs fundamentally from many other games that have pitted humans against AI in ways that embody the difficulty of many real-world situations. Poker players cannot see their opponents’ hands, meaning that, in contrast to checkers, chess, or Go, not all of the information contained within the game is available to them. Researchers from DeepMind, a U.K.-based subsidiary of Alphabet, made headlines last year after creating a program capable of beating one of the world’s best Go players (see “Google’s AI Masters the Game of Go a Decade Earlier Than Expected”).
Heads-up no-limit Texas hold’em is a version of the game played between two people where there is no limit on the amount of money that can be waged. This variant is particularly complex, and for a long time proved too difficult for machines to play expertly. There are 1060 (10 followed by 160 zeros) possible paths of play for each hand in heads-up no-limit Texas hold’em.
DeepStack learned to play poker by playing hands against itself. After each game, it revisits and refines its strategy, resulting in a more optimized approach. Due to the complexity of heads-up no-limit poker, this approach normally involves practicing with a more limited version of the game. The DeepStack team coped with this complexity by applying a fast approximation technique that they refined by feeding previous poker situations into a deep-learning algorithm.
“What's really new for such a complex game is being able to effectively compute the action to take in each situation as it is encountered, rather than having to work through a simplified form of the entire tree of game possibilities offline,” says Wellman of the University of Michigan.
The researchers compare DeepStack’s approximation technique to a human player’s instinct. “This estimate can be thought of as DeepStack’s intuition,” they write. “A gut feeling of the value of holding any possible private cards in any possible poker situation.”
It’s possible to measure the performance of a poker player by looking at the amount won, relative to the amount that the table has bet, over many games. DeepStack had a win rate roughly nine times better than what would be considered good for a professional player.
In 2015, Bowling and colleagues at the University of Alberta “solved” the more limited version of heats up hold’em by developing a poker-bot capable of playing the game perfectly.
The poker-bot involved in the Pittsburgh tournament, called Libratus, was developed by Sandholm and one of his graduate students, Noam Brown. The pair has not yet disclosed details of how their program approaches the game, but Brown says it essentially tries to "solve" the game—or figure out every possible scenario—earlier during the game than was previously possible. Libratus runs on extremely powerful hardware at the Pittsburgh Supercomputing Center, a facility run jointly by CMU and the University of Pittsburgh.
from Poker
Post a Comment