As mentioned previously, an exploitive bot needs to learn how the opposition plays and develop counter strategies to their style. The first thing we need to do to achieve this is gather information on our opposition. There is already software out there to do this for you such and Hold'em Manager, Poker Tracker and Poker Office, you can then tap into their databases. The other option is to create your own database and write a hand importer yourself, the only advantages of this being it's more flexible and you can use a more powerful database engine and tools such as Microsoft Sql Server.
Once you've done this you need hand histories to import, you may have many hand histories of your own, but I'd recommend starting out at the lowest stakes, so unless you've been playing $2 games for a while you're unlikely to have enough. There are various sites online where you can buy hand histories, and several places you can pick up obfuscated hand histories for free.
Once you've got all your hand histories and started talking to your database you can start the process of working with the data. There are two approaches here, a statistics based approach, or a machine learning approach.
The statistics based approach is simplest, it basically says if player x bets in this situation 15% of the time, then we'll assume they're doing this with the top 15% of hands, it's pretty crude, but effective.
With the machine learning approach we're starting to get into the world of Skynet. I'm not going to go into too much detail here, as there is plenty of information on Wikipedia and for more poker specific articles you can take a look at PokerAI.
The general idea behind the machine learning approach is that the computer is going to learn how other players play based on their statistics, and we can use this information to predict how they'll act is certain situations and respond to that. Clever stuff hey?! Machine learning is still very basic however. Artificial neural networks, for example, try to mimic the human brain's method of reinforcement learning. But... the human brain has an estimated 100 billion neurons and is effectively running millions of parallel CPU cores, we'll be working with networks with a number of neurones that you can count on your hands and toes, and unless you have a lot of money, you're not going to be working with millions on parallel CPUs either. So it's a bit like trying to teach poker to a toddler, sometimes they just don't understand!
Good analogy at the end. A statistics based approach can take you pretty far, definitely through the micro-stakes.
ReplyDeleteWell put. One thing-- technically, neural networks don't perform reinforcement learning, not in backprop nets at least. :)
ReplyDeleteI was wondering if it is possible to make a pokerbot work on a distibuted network to increase it's capabilities. Also on a side note I have this Idea of setting up a bot with an architecture similar to that of a distributed network, however this network would be virtualized. Each piece would have it's own separate software depending upon the programming language best fit for the task.
ReplyDeleteA Hand history database server, A probabilities calculator, Simulations and so on and so forth.
All these would be networked together in a virtual environment enabling each section to be completely independent of one another, yet working together as a whole via a virtual intranet. Does this sound like an idea worth seeing through? If Not please explain.
The advantages of distribution is that you can pull together more processing power from multiple machines. The disadvantage is that it takes longer to send a request to another machine / VM that it does to do all processing on one machine. I can see a case for having a distributed system where requests for processing intensive stand alone tasks can be distributed, for example, simulations, but the advantage is lost if you have to make calls too frequently.
ReplyDelete