Wednesday, 11 January 2012

Neural Networks and Genetic Algorithm in trading on a Betting Exchange

This is a report on learning neural networks for trading on betting exchange horse racing markets.

Neural Network [1] was learned using Market Simulator [2] with data collected for a period of 3 months between January and April 2011 for Horse Racing GB win only markets, which allows for replaying betting exchange market on a betting exchange and evaluating trading decisions.

Example of collected market data:
{"time":1282481934744,"eventType":"CREATE_MARKET","marketId":101655582,"marketName":"7f Mdn Stks","eventName":"/GB/Folk 22nd Aug","numOfWinners":1,"marketTime":"2010-08-22 14:10:00","runners": [{"runnerId":4848193,"runnerName":"Dubawi Gulf"},{"runnerId":4818333,"runnerName":"Apace"},{"runnerId":4932434,"runnerName":"Menha"},{"runnerId":4932435,"runnerName":"Flying Arch"},{"runnerId":4883524,"runnerName":"Hello Tomorrow"},{"runnerId":4932436,"runnerName":"Mazagee"}]}
{"time":1282481934744,"eventType":"PLACE_BET","betSize":38.0,"betPrice":5.9,"betType":"LAY","marketId":101655582,"runnerId":4818333}
...
{"time":1282482515736,"eventType":"PLACE_BET","betSize":314.0,"betPrice":1.61,"betType":"BACK","marketId":101655582,"runnerId":4848193}
{"time":1282482515736,"eventType":"CANCEL_BETS","betsSize":5.0,"betPrice":18.0,"betType":"LAY","marketId":101655582,"runnerId":4932435}
{"time":1282482515736,"eventType":"PLACE_BET","betSize":17.0,"betPrice":20.0,"betType":"LAY","marketId":101655582,"runnerId":4932435}
Collected data included market prices, traded volume, volume of unmatched bets by price, current risk position, but no knowledge from historical market data was taken into account such as how good a given horse is comparing to other horses in a race. The output of neural network was either trading action to be taken in a current state, e.g. place bet, cancel bet, or indicator whether a given action, e.g. place bet ( (specified as an input to neural network), is good or not decision.

Neural Network wasn't learned from examples, as those didn't exists on how to trade in HR markets, but rather than that, it was optimized with genetic algorithm [3] and simulated annealing [4]. This type of learning neural networks could be perceived as an example of reinforcement learning [5] in an environment, in which we don't know how to act to achieve the goal, e.g. what trading decisions should be taken, but we know when we reach the goal, in this case it is positive expected profit [6] from trading on a betting exchange markets.

The best instance of learned neural network gives a positive profit per market by average of 5£ including 5% winning commission from trading on HR betting exchange markets.  But it also involves a big number of betting transactions, which causes additional transaction fees and takes overall profit per market from 5£ down to 2£. Adding additional costs around setting up hardware, technology and maintenance, limits the usage of presented trading technique to a research activity only. Involving a knowledge gained from historical data potentially could take this technique from a lab project to production ready system.

Neural network structure

Structure 1:
  • Input neurons - Runner prices, traded volume, unmatched volume, derivatives of price and traded volume, risk position.
  • Hidden layer - Single hidden layer.
  • Output neurons - Actions to be taken, e.g. 3 neurons: A (place back bet), B (place lay bet), C(place hedge bet). 
  • Activation function - Hyperbolic tangent. If output neuron value is higher than 0 then action is taken.
Structure 2:
  • Input neurons -  Runner prices, traded volume, unmatched volume , derivatives of price and traded volume, risk position, a proposal for a single action to be taken, e.g. place back bet.
  • Hidden layer - Single hidden layer.
  • Output neuron - Single neuron indicating how good action specified by an input layer is. Before taking any action, all possible actions are evaluated with neural network and the best one is executed.
The latter neural network produces good results (positive profit) faster than the former one, due to a smaller number of weights inside the neural network to be learned.

Learning process

First, neural network is initialized with an initial set of random weights, then market simulation is executed evaluating trading quality (market expected profit) of an agent over 100-300 markets. The overall expected profit from all markets is a quality value of a neural network. In the next step, neural network weights are updated applying evolutionary algorithm or simulated annealing and simulation process is repeated. A single learning iteration takes about 1 minute and the whole process took weeks till neural network was trained for giving positive profit. Learning was conducted on Intel Core i7 2.8GHz 8 Cores PC.

The best learning results were achieved by applying co-evolution version of learning described above, where a number of traders have to fight each other on a betting exchange markets. With this approach weights of best traders are crossed each other by applying genetic algorithm.

What's next?

In the next step I plan to take advantage of a knowledge from historical market data while learning and then executing trading agents. I'm looking at adopting probability theory, Bayesian networks [7], logistic regression [8] and other techniques suitable for analysing huge amount of historical market data.

Notes

A number of trading examples examined for a purpose of this research work are available here:
http://code.google.com/p/betting-ai/source/browse/#svn%2Ftrunk%2Ftrader-examples

All software for a purpose of this project is written in a Scala programming language.

References
  1. Neural Network - http://en.wikipedia.org/wiki/Neural_network
  2. Market Simulator -  http://blog.danmachine.com/2010/08/horse-racing-market-simulation.html
  3. Genetic algorithm -  http://en.wikipedia.org/wiki/Genetic_algorithm
  4. Simulated annealing -  http://en.wikipedia.org/wiki/Simulated_annealing
  5. Reinforcement learning -  http://en.wikipedia.org/wiki/Reinforcement_learning
  6. Expected value -  http://en.wikipedia.org/wiki/Expected_value
  7. Bayesian networks -  http://en.wikipedia.org/wiki/Bayesian_networks
  8. Logistic regression -  http://en.wikipedia.org/wiki/Logistic_regression