Thursday, 25 November 2010

Analysing Betting Exchange markets with Complex Event Processing, Esper and Scala

Recently, I posted on analysing betting exchange markets with betting market simulator and time line charts (read more...) At that time, I demonstrated charts with runner price that don't present a significant analytical value, except of being  a good example to introduce charting feature of betting market simulator. This day, I present how to analyse betting exchange markets with Esper complex event processing tool and Scala. This should be more useful in practice than studying simple metrics such as runner price or traded volume.

Complex event processing is a broad concept. I use it for analysing different types of market data such as price, probability, traded volume, risk metrics, etc. Complex event processing allows me to combine all those metrics altogether and prepare custom analysis  quickly and with minimal programming effort. I found it useful in both real time event processing as well as in historical analyses. There are many complex event processing tools on the market, both commercial and open source. The one I use is Esper, an open source tool that makes complex event processing simple in Java programming  language.

One of core features of Esper is an ability to create sliding windows and then, on top of that, to calculate some metrics such as avg, min, max or recognise patterns such as correlation between runner price and runner traded volume. Additionally, sliding windows may be used to calculate derivatives, which is an example that I will present in this post. Basically, I will demonstrate how to calculate first derivative of runner traded volume with respect to time and then how to calculate the second derivative. I'm finding those metrics useful to analyse correlation between market runner implied probability and traded volume for betting markets.

Below I present three charts for horse racing market during 10 minutes before market is turned in play. The first chart shows implied probability (1/runner price) for all market runners, for that, no complex event processing is required. The second chart presents the first derivative of traded volume with respect to time for all market runners, where as the last one displays the second derivative of runner traded volume.

To create a chart with implied probabilities for all market runners, we need to iterate through all market runners, then calculate runner implied probability and add it to chart values that will be used to generate time  line chart after simulation with Market Simulator tool is finished:

-------------------------------------------------------------------------------------------------------------
// For all market runners add implied probability to time line chart
for(runnerId <- ctx.runners.map(_.runnerId)) {

  //get best back and lay prices for market runner
  val bestPrices = ctx.getBestPrices(runnerId)
  
  // calculate average price between best back and lay prices
  val avgPrice = PriceUtil.avgPrice(bestPrices._1.price -> bestPrices._2 .price)

  
  //add average prices to chart values
  ctx.addChartValue(runnerId.toString, 1/avgPrice)
}
-------------------------------------------------------------------------------------------------------------

Figure 1 Implied probability for all market runners, y_axis - traded volume, x_axis - time

In order to display derivative of runner traded volume with respect to time we use the following formula:

y(v1,v0,t1,t0) = (v1 - v0) / (t1-t0) where
  • y - derivative of runner traded volume with respect to time
  • v1- runner traded volume at time t1
  • v0 - runner traded volume at time t0
  • t1 - time t1, in this example t1-t0 = 120sec
  • t0 - time t0
To calculate this function value  for every second during last 10 minutes before horse racing market is turned in play, a sliding window is a way to go. Having a sliding window available for every second with a size of 120sec, we can obtain the first and the last values of traded volume in that window. The last value is a traded volume at time t1, whereas the first value is a traded volume at time t0, which is t1-120sec. All that to work requires just a bit of Esper declarative configuration.

First of all, we need to send a timestamped event to Esper engine with trader volume for a market runner:

-------------------------------------------------------------------------------------------------------------
//Create market event as a map with three attributes: runnerId, traded volume and timestamp
val event = Map("runnerId" -> runnerId,"tradedVolume" -> ctx.getRunnerTradedVolume(runnerId).totalTradedVolume,"timestamp"-> ctx.getEventTimestamp/1000)

//Send event to Esper
epService.getEPRuntime().sendEvent(event, "TradedVolumeEvent")
-------------------------------------------------------------------------------------------------------------

then we create a sql like query to calculate derivative value and register it with Esper engine. One thing worth to mention is the last part of this query 'group by runnerId'. Events have to be grouped by runner id, because events for many market runners are processed.

-------------------------------------------------------------------------------------------------------------
val expression = "select runnerId, (last(tradedVolume)-first(tradedVolume)) / (last(timestamp)-first(timestamp)) as delta from TradedVolumeEvent.win:time(120 sec) group by runnerId"

val statement = epService.getEPAdministrator().createEPL(expression)
-------------------------------------------------------------------------------------------------------------

The value of this query is calculated every time when new event is added to Esper engine, a simple listener updates time line chart with derivative value:

-------------------------------------------------------------------------------------------------------------
statement.addListener(new UpdateListener {
  def update(newEvents: Array[EventBean], oldEvents: Array[EventBean] ) {
    val delta = newEvents(0).get("runnerId"),newEvents(0).get("delta").asInstanceOf[Double]
    ctx.addChartValue("tv" + delta)
  }
})
-------------------------------------------------------------------------------------------------------------

Scala class to calculate derivative of traded volume is here.

Figure 2 Derivative of traded volume for all market runners with respect to time

Calculating the second derivative of runner traded volume requires two phases. First of all, the first derivative of traded volume is calculated and added to Esper engine as an event:

-------------------------------------------------------------------------------------------------------------
insert into TradedVolumeDeltaEvent select runnerId,timestamp,(last(tradedVolume)-first(tradedVolume))/(last(timestamp)-first(timestamp)) as tradedVolumeDelta from TradedVolumeEvent.win:time(120 sec) group by runnerId
-------------------------------------------------------------------------------------------------------------

Then the second derivative is calculated using sql like query:

-------------------------------------------------------------------------------------------------------------
select runnerId,(last(tradedVolumeDelta)-first(tradedVolumeDelta))/(last(timestamp)-first(timestamp)) as tradedVolumeDeltaPrim from TradedVolumeDeltaEvent.win:time(120 sec) group by runnerId
-------------------------------------------------------------------------------------------------------------

Scala class to calculate the second derivative of traded volume is here.

Figure 3 Second derivative of traded volume for all market runners with respect to time.

There are junit tests for all three scala classes that I'm referring to (here), so you can run them yourself if you want to play with examples on complex event processing that I presented in this post. Those junit tests generate the same html graphs as above into a target directory. Eventually please use maven command 'mvn clean install' inside http://code.google.com/p/betting-ai/source/browse/trunk/trader-examples/ to run all junit tests with one call.

References:
  1. Complex Event Processing - http://en.wikipedia.org/wiki/Complex_event_processing
  2. Esper - http://esper.codehaus.org/
  3. Scala - http://www.scala-lang.org/
  4. Betting Market Simulator - http://code.google.com/p/betting-ai/

Friday, 22 October 2010

Market analysis with time line charts

In my last post I was writing about analysing trading strategies with a market simulator. This time I'd like to present a charts feature that makes it easier to analyse trading strategies. Before I start, a bit of information for those who read my blog for the first time.

The market simulator I'm referring to, allows for betfair market simulations that produce results very close to running trading strategies on real betfair markets. It's due to replaying market in a form of bet placement events on a real betting exchange engine.

Time line charts can present any data including standard metrics such as price, probability, traded volume, customer ifwin, customer expected liability, etc, as well as any transformation of those basic metrics, e.g. math functions such as derivativess, statistical, etc. I present below three chart examples. The first chart represents probability for all market runners during last 10 minutes before market is turned in play.


1. Probability for all market runners

Where as the second and third charts present market expected profit for two different trading strategies. As you can see, one of them generates profit, while the next one generates loss. 

 2. Market current expected for a market (profit)

  3. Market current expected for a market (loss)

One more thing worth to mention is a great flexibility that makes it possible to presents aggregated metrics across both time and many markets. What metrics are presented is up to a trader implementation class. Consider the trader implementation below:

def execute(eventTimestamp:Long,ctx: ITraderContext)= {
  if(bestPrices._2.price>2)
    ctc.placeBet(2,bestPrices._2.price,BACK,runnerId)
  ctx.addChartValues(ctx.risk.ifWin)
}

There is a method on a trader context named ctx.addChartValues(...), that adds timestamped metrics to be displayed on a chart after simulation is finished. Let say we want to present log(ifWin), then just call:

ctx.addChartValues(log(ctx.risk.ifWin))

References

Friday, 27 August 2010

Horse racing market simulation

At this point of time, the Betting AI provides both market data collector and market simulator tools. Market data collector provides history of a market on a betting exchange in a form of market events, where as market simulator allows for testing trading strategies based on a historical market data. After simulation is finished the following report is generated (simulation for one day on UK horse racing markets, 21 markets):

***********************************************************************************

*Market Simulator Copyright 2010 Daniel Korzekwa(http://danmachine.com)           *
*Project homepage: http://code.google.com/p/betting-ai/                           *
*Licenced under Apache License 2.0(http://www.apache.org/licenses/LICENSE-2.0)    *
***********************************************************************************

Simulation is started. Simulation progress: 0% 4% 9% 14% 19% 23% 28% 33% 38% 42% 47% 52% 57% 61% 66% 71% 76% 80% 85% 90% 95% 100%
Simulation is finished in 65907 milliseconds.

Expected profit report for trader dk.trader.Trader:
7f Mdn Stks: /GB/Folk 22nd Aug minProfit/prob=-92.58/0.63 maxProfit/prob=345.0/0.01 expProfit=9.05 expAggrProfit=9.05 mBets=128 uBets=7
2m6f Hcap Hrd: /GB/Newt 22nd Aug minProfit/prob=-173.4/0.14 maxProfit/prob=80.4/0.14 expProfit=1.84 expAggrProfit=10.89 mBets=23 uBets=4
1m6f Hcap: /GB/Muss 22nd Aug minProfit/prob=-1182.9/0.19 maxProfit/prob=522.0/0.06 expProfit=24.83 expAggrProfit=35.73 mBets=226 uBets=17
1m Hcap: /GB/Muss 22nd Aug minProfit/prob=-1036.8/0.27 maxProfit/prob=615.0/0.01 expProfit=-11.99 expAggrProfit=23.74 mBets=222 uBets=9
3m2f Hcap Chs: /GB/Newt 22nd Aug minProfit/prob=-968.55/0.21 maxProfit/prob=1153.65/0.18 expProfit=-25.4 expAggrProfit=-1.66 mBets=184 uBets=12
2m5f Nov Chs: /GB/Newt 22nd Aug minProfit/prob=-1812.6/0.24 maxProfit/prob=1659.15/0.33 expProfit=-105.13 expAggrProfit=-106.8 mBets=468 uBets=14
2m3f Nov Hrd: /GB/Newt 22nd Aug minProfit/prob=-729.3/0.16 maxProfit/prob=738.36/0.31 expProfit=-3.33 expAggrProfit=-110.13 mBets=178 uBets=10
5f Mdn: /GB/Folk 22nd Aug minProfit/prob=-932.6/0.29 maxProfit/prob=591.4/0.11 expProfit=-8.61 expAggrProfit=-118.74 mBets=233 uBets=12
1m Hcap: /GB/Muss 22nd Aug minProfit/prob=-1687.7/0.16 maxProfit/prob=1637.55/0.34 expProfit=46.21 expAggrProfit=-72.53 mBets=421 uBets=24
7f Sell Stks: /GB/Folk 22nd Aug minProfit/prob=-3.0/0.09 maxProfit/prob=7.65/0.28 expProfit=0.07 expAggrProfit=-72.46 mBets=3 uBets=5
5f Sell Stks: /GB/Muss 22nd Aug minProfit/prob=-1699.8/0.28 maxProfit/prob=1462.06/0.3 expProfit=-112.54 expAggrProfit=-185.0 mBets=384 uBets=11
2m6f Hcap Hrd: /GB/Newt 22nd Aug minProfit/prob=-1088.4/0.2 maxProfit/prob=454.8/0.08 expProfit=-8.51 expAggrProfit=-193.51 mBets=156 uBets=8
2m1f NHF: /GB/Newt 22nd Aug minProfit/prob=-1068.0/0.2 maxProfit/prob=1105.8/0.29 expProfit=38.09 expAggrProfit=-155.42 mBets=230 uBets=16
7f Mdn Stks: /GB/Muss 22nd Aug minProfit/prob=-822.6/0.45 maxProfit/prob=1197.9/0.1 expProfit=-26.43 expAggrProfit=-181.85 mBets=333 uBets=12
1m4f Hcap: /GB/Muss 22nd Aug minProfit/prob=-750.72/0.4 maxProfit/prob=561.0/0.03 expProfit=-13.69 expAggrProfit=-195.55 mBets=207 uBets=7
7f Hcap: /GB/Folk 22nd Aug minProfit/prob=-849.45/0.25 maxProfit/prob=471.0/0.07 expProfit=-20.03 expAggrProfit=-215.57 mBets=167 uBets=5
5f Hcap: /GB/Muss 22nd Aug minProfit/prob=-564.3/0.18 maxProfit/prob=219.0/0.04 expProfit=7.55 expAggrProfit=-208.02 mBets=102 uBets=5
6f Hcap: /GB/Folk 22nd Aug minProfit/prob=-1268.7/0.2 maxProfit/prob=1548.0/0.16 expProfit=35.25 expAggrProfit=-172.78 mBets=361 uBets=22
2m1f Mdn Hrd: /GB/Newt 22nd Aug minProfit/prob=-922.5/0.18 maxProfit/prob=684.15/0.23 expProfit=53.31 expAggrProfit=-119.47 mBets=280 uBets=18
1m1f Hcap: /GB/Folk 22nd Aug minProfit/prob=-875.91/0.32 maxProfit/prob=2577.9/0.16 expProfit=-19.39 expAggrProfit=-138.86 mBets=489 uBets=20
1m4f Hcap: /GB/Folk 22nd Aug minProfit/prob=-589.23/0.35 maxProfit/prob=1018.5/0.22 expProfit=-10.18 expAggrProfit=-149.05 mBets=249 uBets=11
------------------------------------------------------------------------------------
TotalExpectedProfit=-149.05 TotalMatchedBets=5044 TotalUnmachedBets=249


The list below presents a few market events for a horse racing market:

{"time":1282481934744,"eventType":"CREATE_MARKET","marketId":101655582,"marketName":"7f Mdn Stks","eventName":"/GB/Folk 22nd Aug","numOfWinners":1,"marketTime":"2010-08-22 14:10:00","runners": [{"runnerId":4848193,"runnerName":"Dubawi Gulf"},{"runnerId":4818333,"runnerName":"Apace"},{"runnerId":4932434,"runnerName":"Menha"},{"runnerId":4932435,"runnerName":"Flying Arch"},{"runnerId":4883524,"runnerName":"Hello Tomorrow"},{"runnerId":4932436,"runnerName":"Mazagee"}]}
{"time":1282481934744,"eventType":"PLACE_BET","betSize":38.0,"betPrice":5.9,"betType":"LAY","marketId":101655582,"runnerId":4818333}
...
{"time":1282482515736,"eventType":"PLACE_BET","betSize":314.0,"betPrice":1.61,"betType":"BACK","marketId":101655582,"runnerId":4848193}
{"time":1282482515736,"eventType":"CANCEL_BETS","betsSize":5.0,"betPrice":18.0,"betType":"LAY","marketId":101655582,"runnerId":4932435}
{"time":1282482515736,"eventType":"PLACE_BET","betSize":17.0,"betPrice":20.0,"betType":"LAY","marketId":101655582,"runnerId":4932435}

Example of trader implementation:

/**Executes trader implementation so it can analyse market on a betting exchange and take appropriate bet placement decisions.
*
* @param ctx Provides market data and market operations that can be used by trader to place bets on a betting exchange market
*/
def execute(ctx: ITraderContext) = {

for(runner <- ctx.runners) {
val bestPrices = ctx.getBestPrices(runner.runnerId)

if(bestPrices._1>2) {
ctx.placeBet(2,bestPrices._1,BACK,runner.runnerId)
ctx.placeBet(2,bestPrices._1 - 0.02,LAY,runner.runnerId)
}
if(bestPrices._2<2.2) {
ctx.placeBet(2,bestPrices._2,LAY,runner.runnerId)
ctx.placeBet(2,bestPrices._2 + 0.02,BACK,runner.runnerId)
}
}
}

Project home page
http://code.google.com/p/betting-ai/

Wednesday, 7 July 2010

The BettingAI - Market Data Collector Design

Background


Trading analyst wants to analyse trading strategies using historical market data and the Market Simulator tool. For the purpose of that, a real market data for markets on the Betfair betting exchange has to be collected.

Vision


The vision is to create a standalone application that is collecting market data until  it is terminated by a user. Collected market data is written to files on a file system (one file per market) in a format that is supported by the Market Simulator tool

The following input parameters for the application are provided:
  • Betfair account credentials: user name, password and product id.
  • Time interval of the market data collection. 

The market data should be collected for:
  • Horse racing win only markets.
  • The time period starting 10 minutes before market start time until market is turned in play.

Requirements


As a trading analyst.
I want to collect market data for markets on the Betfair betting exchange.
So that I can run market simulation and test trading strategy for a real market data.

Acceptance criteria:
  • Market data is collected in a format that is support by the MarketSimulator ver 1.0
  • Market data is stored in text files (one file per one market)
  • Market data is collected for all Horse Racing win only markets.
  • Market data is collected starting 10 minutes before market start time until it's turned in play.
  • Installation package supported by both Windows and Linux operating systems is provided.
  • Command line interface is provided with the following input parameters: 
    • Betfair account credentials: user name, password and product id.
    • Time interval of the market data collection.

    User interface


    As a trading analyst I want to start market data collector from the command line interface as presented below:

    MarketDataCollector bfUser=john bfPassword=bravo6 bfProductId=82 collectionInterval=1

    • user name, password and product id - Credentials for the Betfair account that is used for market data collection. 
    • collectionInterval [seconds] - The time interval for the market data collection.

    [BEGINNING OF THE OUTPUT]
    Market data collection is started.
    [THE END OF THE OUTPUT]

    Component design




    • Market Data Collector Launcher - Parses input parameters and starts scheduler for Market Data Collector Task.

    • Market Data Collector Task - Gets market events for all monitored markets and stores them in files on a file system (one file per market).

    • Market Event Producer
      • First of all, get list of all monitored markets from the betfair api. Then for each market do the following:

      • Get market prices and total traded volume from the Betfair API.

      • Calculate market events that represent the delta between previous and current state of the market. If no previous market state is available then CREATE_MARKET event will be generated as the first on the list.

      • Add those market events to the Betex (Implementation of a betting exchange), then get market prices and total traded volume for the market and compare both data against market prices and total traded volume that was received from the Betfair API. The reason for that is to check if market events have been generated correctly. If both data don't match then exception should be thrown.

      • After all markets are processed, market events for all market are returned.

    • Market Event Calculator - Calculates market events that represent the delta between previous and current state of the market. If no previous market state is available then CREATE_MARKET event will be generated as the first event on the list.

    Deployment design




    • Java VM - The Betting AI application is written in the Scala programming language that is running within the Java virtual machine.

    • Scala - The scala language distribution, contains scala libraries that are required to run scala applications.

    • Market Data Collection - Executables of the Market Data Collector, contains start script to run market data collection.

    • File system - Market Data Collector is storing collected market data on a files system (one file per market).

    • Betfair API WS - This is Betfair public API that is used to collect market data.

    Monday, 19 April 2010

    The Betting AI (Artificial intelligence) - domain model, components, deployment

    Previous part of the betting ai design.

    Domain model


    • Betting Exchange - "...is a form of bookmaking in which the operator offsets its risk perfectly through technology, such that the effect to the customer is that customers are seen to bet between themselves..." - from Wikipedia.
    • Market - "...The market is where you place a bet on an event. Any given event, such as a soccer match, may have a number of different markets: for example, Match Odds, a Handicap and First Goalscorer markets. This is the place to view the odds and have a bet..." - from Betfair help.
      • market id - Unique  market id.
      • market name - e.g. Match Odds or Place market
      • event name - e.g. Mat Utd vs Arsenal
      • market state - active/in play/settled
      • selection id - Unique selection id, e.g. 100 for Manchester United and 101 for Arsenal.
      • selection name - e.g. Arsenal
    • Bet - "...A ‘Back’ bet is a bet on something to happen – a horse to win a race, a football team to win, a person to be evicted from Big Brother etc. It’s just like the bet you’d regularly place with a traditional bookmaker or tote, or even with your mates..." - from Betfair help. "...Laying a bet means you are betting on something not to happen – a match not to end in a draw, a horse not to come in first, a rugby team not to win by over 7 pts etc..." - from Betfair help
      • bet price - Bet odds. Chances of winning written in a decimal format = 1/probability, e.g. 1/0.5 = 2 ( fifty-fifty).
      • bet size, e.g. 20£
      • bet type - Back or lay.
      • market id/selection id - Unique id of a particular selection in a given market.
      • bet status - Unmatched, matched, cancelled (by user), lapsed (by a betting exchange, e.g. when market is turned in play), settled.
    • Historical market data - Represents the whole market life cycle: market creation, bet placement, turning market in play, bet placement in play and market settlement. It allows to replay a market from the past.
    • Trader - Responsible for analysing market and taking appropriate bet placement decision. The aim of bet placement is to guarantee the highest profit with the lowest risk. There are two types of traders, a real trader that is represented by human and artificial trader that is represented by computer software. The main aim of this project is to provide a simulation environment for testing artificial traders.
    • Expected profit - This is a measure that tells, what would be the profit in a long term if the same market was processed many times. For instance if the expected market profit is 10, then it means that for 100 'exactly the same' markets the profit would be 1000. It may happen that for the first market the profit will be -23 and for the next market it will be 12, but in total, based on a theory of probability it should be approximately 1000.

    Component design



    Components: 
    • User - A person who is using the market simulation. Usually it's a market analyst or a real trader.
    • Simulator App - Provides a command line interface to the market simulation. Read the configuration of the historical market data and the trader implementation, runs simulation and displays expected profit report on the screen.
    • Simulator - Does exactly the same as a simulator application, but provides application programming interface instead of command line interface.
    • Market Event Processor - Parses historical market event in a string format, e.g. CREATE_MARKET and calls appropriate method on a betting exchange, e.g. createMarket(market:Market)
    • Trader - Analyses markets on a betting exchange and places bets.
    • Expected Profit Calculator - Reads matched bets for user (e.g. trader) from a betting exchange and calculates expected market profit report.
    • Betex - It's a betting exchange that provide peer-to-peer betting services.

    Flow of control:
    1. User starts the market simulator giving the path to the file with the historical market data and the name of the trader implementation class. After the market simulation is finished the market expected profit report for the trader is printed on the screen.
    1. The Simulator Application calls the Simulator component with the input parameters provided by the user.
    1. The Simulator component reads the file with historical market data and for each historal market event, calls both MarketEventProcessor and Trader. After all historical market events are processed, it calls the ExpectedProfitCalculator to get the market expected profit report.

    Deployment design


    • Java VM - The Betting AI application will be written in the Scala programming language that is running within the Java virtual machine.
    • Scala - The scala language distribution, contains scala libraries that are required to run scala applications.
    • Betting AI application - Binaries of the betting ai application, contains start script to run market simulation.
    • Historical market data file - This file contains historical market data that is used for market simulation.
    • Trader implementation class - This file contains trader implementation that market simulation is executed for.

    References

    Project home page: http://code.google.com/p/betting-ai/

    Wednesday, 14 April 2010

    The Betting AI (Artificial intelligence) - user interface and input data

    Previous part of the betting ai design.
    Next part of the betting ai design.

    User interface

    As a trading analyst I want to run the market simulation from a command line interface as presented below:

    app_name marketData=[market_data_file] traderImpl=[trader_impl_class]

    • app_name - Name of the application, e.g. market_simulator

    • marketData - Text file with market data that will be used for the simulation. Go to the next chapter 'Input data' for more details.

    • traderImpl - Fully classified name of the trader implementation class that the simulation will be executed for. Go to the next chapter 'Input data' for more details.

    After simulation is started, the following output is presented on the screen:

    [BEGINNING Of THE OUTPUT]

    Simulation is started.
    Simulation progress: 1% 2% 3% 4% 5% 6% ....................................................... .................................................................................................................
    ..................................100%
    Simulation is finished in 23 sec.

    Expected profit report for trader com.dk.bettingai.trader.SimpleTraderImpl:

    Catt 7th Apr: 14:10 7f Claim Stks    expProfit=28 expAggrProfit=28  mBets=34 uBets=17
    Catt 7th Apr: 14:40 1m6f Hcap       expProfit=4   expAggrProfit=32  mBets=27 uBets=2
    ...
    Nott 7th Apr: 17:20 1m Hcap         expProfit=-6  expAggrProfit=136 mBets=45 uBets=57
    -------------------------------------------------------------------------------------
    TotalExpectedProfit=136 TotalMatchedBets=457 TotalUnmachedBets=262
    [END Of THE OUTPUT]

    Definitions:
    • expProfit - Expected market profit based on all matched bets on the market placed by trader.

    • expAggrProfit- Expected profit for the market and all prior markets. 

    • mBets - Number of matched bets on the market placed by trader.

    • uBets - Number of unmatched bets on the market placed by trader.

    • TotalExpectedProfit - Expected profit for all markets and all matched bets placed by trader.

    • TotalMatchedBets - Number of all matched bets for all markets placed by trader.

    • TotalUnmatchedBets - Number of all unmatched bets for all markets placed by trader.

    Input data

    To run the market simulation, user has to provide the historical market data and the trader implementation class.

    The historical market data is provided as a text file, that contains the number of events for the market management and the bet placement. The list below presents all possible event types:
    • CREATE_MARKET
      • Market id, market name, event name, number of winners, market time.
      • List of selections: selection id, selection name.

    • TURN_MARKET_INPLAY
      • Market id.

    • SETTLE_MARKET
      • Market id, list of selection ids for winners.

    • PLACE_BET
      • Bet size, bet price, bet type (back/lay), market id, selection id.

    • CANCEL_BET
      • Bet id.

    The trader implementation class implements the following interface:

    Interface name: Trader
    Interface operations: 
    • execute():void

    • setBetexPublicApi(betexPublicApi: BetexPublicApi):void

    BetexPublicApi:
    • getActiveMarkets
      • Input - None.
      • Output -  All markets that are not closed. The following attributes are returned: market id, market name, event name, market time.

    • getMarketData
      • Input - Market id.
      • Output - All back and lay stakes for each price on the exchange for a given market. 
      • Output - Market id, market name, event name, selection ids/names, number of winners, market status (active/in play/settled), market time.
      • Output - matched amounts at each price on all of the runners.

    • placeBet - Place a bet on a betting exchange. 
      • Input: Bet size, bet price, bet type (back/lay), market id, selection id

    • cancelBet - Cancel a bet.
      • Input - Bet id.
      • Output - None.

    • getBets - Returns bets for a given user on a betting exchange
      • Input: User id, bet status (matched, unmatched, matched_unmatched, settled, cancelled, lapsed)
      • Output: List of bets (bet id, bet size, bet price, bet type, market id, selection id, bet status).

    References


      Friday, 9 April 2010

      The Betting AI (Artificial intelligence) - requirements

      Previous part of the betting ai design.
      Next part of the betting ai design.

      Functional requirements

      Use case scenario:
      As a trading analyst

      I want to choose historical market data for the simulation, e.g. all market data for the period from 01.02.2010 to 10.02.2010

      • Historical market data represents the whole market life cycle: market creation, bet placement, turning market in play, bet placement in play and market settlement.

      Then I want to choose trader implementation that the bet placement intelligence should be tested for. 

      • Trader implementation should be executed every time the historical market event is processed. For instance when market is created or when bet is placed or market is turned in play. When trader is executed it can check the market state, runner prices, etc. and take appropriate trading decisions.

      Then I want to run simulation.

      • The aim of the simulation is to test, how a trader implementation would behave if it was running live on the betting exchange. Of course the market simulation will never return the same results as testing trader on a real betting exchange, however they both should be similar as far as possible.


      Finally I want to see expected trader profit based on matched bets placed by trader and market probabilities derived from market prices before market is turned in play.
      • Expected trader profit for market tells us what would be the profit in a long term if this market was processed many times. For instance if the expected market profit is 10, then it means that for 100 exactly the same markets the profit would be 1000. It may happen that for the first market the profit will be -23 and for next market it will be 12, but in total, based on a theory of probability it should be approximately 1000.


      Non-functional requirements

      • Performance - Running the market simulation for 10 markets with 10 runners each and collected data for 10 minutes with 1 second interval should take maximum 1 second.

      • Command line interface - The market simulation should be started from the command line interface.

      • Multi-platform - It should be possible to run the market simulation on Windows and Linux operating systems.

      • Historical market data - All historical market data for the market simulation should be read from a single file on a file system. 

      • Installation - The application for the market simulation should be delivered as a zip file. To install and run the market simulation the following steps should be required:
        • Unzip the application.
        • Go to the application folder.
        • Run start script and provide file with the historical market data and the trader implementation.
        • Print the simulation report to the output screen.

      • Easy to run the market simulation for different traders - Running the market simulation for different traders should require only to change the name of the trader provided on the command line interface.

      References

      Monday, 5 April 2010

      The Betting AI (Artificial intelligence) - project initiation

      Next part of the betting ai design.

      Background

      In 2008 I started a research project that was to analyse and understand behaviours on a betting exchange. On the beginning of my research I created FlexiBet, automated betting system based on state machines. For the purpose of the Flexibet project I developed betting-scxml, domain specific language for creating automated betting strategies based on SCXML state machines. 

      Having that, I created a few betting strategies and I put them live to get some data to analyse. In the next step I analysed collected data, then I created new betting strategy and I put it live again to get more data to analyse.

      I was evolving betting strategies and analysing collected data for one year or so, until it became time consuming and difficult to continue. It was because of using manual data analysis, mostly based on SQL and Eclipse Birt Business Intelligence. Despite of being simple and handy for creating simple reports, I found it difficult to analyse complex data.

      Vision

      The vision is to apply neural network technique to analyse markets on a betting exchange. I want to create a trader based on a neural network that can take appropriate bet placement decisions. Then, trader should be tested on a historical market data to measure how efficient the trader's intelligence is.

      Use case scenario:
      I want to choose historical market data for the simulation, e.g. all market data for the period from 01.02.2010 to 10.02.2010

      Then I want to choose trader implementation that the bet placement intelligence should be tested for.

      Then I want to run simulation.

      Finally I want to see expected trader profit based on matched bets placed by trader and market probabilities derived from market prices before market is turned in play.

      References

      Project home page: http://code.google.com/p/betting-ai/