Wednesday, 7 July 2010

The BettingAI - Market Data Collector Design

Background


Trading analyst wants to analyse trading strategies using historical market data and the Market Simulator tool. For the purpose of that, a real market data for markets on the Betfair betting exchange has to be collected.

Vision


The vision is to create a standalone application that is collecting market data until  it is terminated by a user. Collected market data is written to files on a file system (one file per market) in a format that is supported by the Market Simulator tool

The following input parameters for the application are provided:
  • Betfair account credentials: user name, password and product id.
  • Time interval of the market data collection. 

The market data should be collected for:
  • Horse racing win only markets.
  • The time period starting 10 minutes before market start time until market is turned in play.

Requirements


As a trading analyst.
I want to collect market data for markets on the Betfair betting exchange.
So that I can run market simulation and test trading strategy for a real market data.

Acceptance criteria:
  • Market data is collected in a format that is support by the MarketSimulator ver 1.0
  • Market data is stored in text files (one file per one market)
  • Market data is collected for all Horse Racing win only markets.
  • Market data is collected starting 10 minutes before market start time until it's turned in play.
  • Installation package supported by both Windows and Linux operating systems is provided.
  • Command line interface is provided with the following input parameters: 
    • Betfair account credentials: user name, password and product id.
    • Time interval of the market data collection.

    User interface


    As a trading analyst I want to start market data collector from the command line interface as presented below:

    MarketDataCollector bfUser=john bfPassword=bravo6 bfProductId=82 collectionInterval=1

    • user name, password and product id - Credentials for the Betfair account that is used for market data collection. 
    • collectionInterval [seconds] - The time interval for the market data collection.

    [BEGINNING OF THE OUTPUT]
    Market data collection is started.
    [THE END OF THE OUTPUT]

    Component design




    • Market Data Collector Launcher - Parses input parameters and starts scheduler for Market Data Collector Task.

    • Market Data Collector Task - Gets market events for all monitored markets and stores them in files on a file system (one file per market).

    • Market Event Producer
      • First of all, get list of all monitored markets from the betfair api. Then for each market do the following:

      • Get market prices and total traded volume from the Betfair API.

      • Calculate market events that represent the delta between previous and current state of the market. If no previous market state is available then CREATE_MARKET event will be generated as the first on the list.

      • Add those market events to the Betex (Implementation of a betting exchange), then get market prices and total traded volume for the market and compare both data against market prices and total traded volume that was received from the Betfair API. The reason for that is to check if market events have been generated correctly. If both data don't match then exception should be thrown.

      • After all markets are processed, market events for all market are returned.

    • Market Event Calculator - Calculates market events that represent the delta between previous and current state of the market. If no previous market state is available then CREATE_MARKET event will be generated as the first event on the list.

    Deployment design




    • Java VM - The Betting AI application is written in the Scala programming language that is running within the Java virtual machine.

    • Scala - The scala language distribution, contains scala libraries that are required to run scala applications.

    • Market Data Collection - Executables of the Market Data Collector, contains start script to run market data collection.

    • File system - Market Data Collector is storing collected market data on a files system (one file per market).

    • Betfair API WS - This is Betfair public API that is used to collect market data.