Poker Neural Nets
The neural network is a weighted graph where nodes are the neurons and the connections are represented by edges with weights. It takes input from the outside world and is denoted by x(n). Each input is multiplied by its respective weights and then they are added.
Hey jp, great points. I would have posted the code, but for the life of me I can't find it anywhere. I had the Excel sheet stored in a different folder, which is why I was able to post it.- Neural network jargon. activation: the output value of a hidden or output unit. epoch: one pass through the training instances during gradient descent. transfer function: the function used to compute the output of a hidden/ output unit from the net input.
- The neural network in a person's brain is a hugely interconnected network of neurons, where the output of any given neuron may be the input to thousands of other neurons. Learning occurs by repeatedly activating certain neural connections over others, and this reinforces those connections.
Anyway, I think I was thinking that including the difference from the average value would help the neural network identify pairs. In retrospect, it should have been able to recognize that simply from the card values, no?
Limit holdem had been done a lot and No Limit was my area of expertise, which is why I chose it. I didn't have that much experience with shortstacking, but I had read pretty much everything there was to read on it and it seemed like a good start because of its simplicity. Little did I know that the profits from shortstacking come from a very few carefully timed moves that are close to impossible for a rules-based bot to emulate (more on that in another post). Eventually I decided to build a no limit heads up bot, which is where my success wound up being.
I didn't mention it in the post, but the key was moving away from rules based to a value based system, where the bot calculated the profitability of its options and decided based on that. It was a lot easier to debug, but there were a whole set of challenges to that too, like estimating your opponent's range.
DeepStack, one of many recent computers to face off against human beings, defeated 11 professional poker players in heads-up no-limit hold’em, according to a study published in Science this month.
Of the 11 players, DeepStack defeated 10 of them in December 2016 by statistically significant margins after the study authors had the computer undergo deep learning training to teach the bot to develop poker intuition for any situation.
The computer looked up two copies of the same network in its neural network, namely for the first three shared cards and then again for the final two, trained on 10,000 randomly drawn poker games, reported Ars Technica.
The researchers recruited 33 players through the International Federation of Poker.
Only 11 players finished 3,000 matches over the course of a four-week period and DeepStack’s neural networks were what allowed it to essentially “learn” and model higher-level concepts while it ran on a gaming laptop (NVIDIA GTX 1080). DeepStack was developed by researchers at the University of Alberta and a number of Czech universities.
DeepStack works through situations as humans would, learning pieces of the game as it goes and create a strategy to defeat the humans.
“In some sense this is probably a lot closer to what humans do,” said Michael Bowling, professor of machine learning and the study author, to Scientific American. “Humans certainly don’t, before they sit down and play, precompute how they’re going to play in every situation. And at the same time, humans can’t reason through all the ways the poker game would play out all the way to the end.”
Poker Neural Network Github
DeepStack isn’t the only artificial intelligence out there. Carnegie Mellon’s Libratus recently beat four professional players with a more elite status on a supercomputer. Its technology is similar to that of DeepStack in the later stages of computing but it does not use the same neural networks, according to Scientific American. DeepStack also won by larger margins.
Past attempts with Claudico didn’t pan out, but Google DeepMind’s Alpha Go beat pros at the game, go. Even more notably, 20 years ago, Deep Blue beat World Chess Champion Garry Kasparov at his own game.
Poker Neural Network
This finding reveals a lot about artificial intelligence’s ability to master imperfect information games beyond abstraction (or computing how to play in every situation before the game begins).
Lead image courtesy of Thigala shri/Flickr
Poker Neural Network
Tags
Poker PlayersAI