New Betfair Historical Data
Ive downloaded the historical data (free version) but am not sure how to work with it. Ideally I could get it into excel somehow. Does anyone know how I would be able to analyze/back test strategies using this data? Any guidance on how the data is arranged is greatly appreciated!
-
ShaunWhite
- Posts: 9731
- Joined: Sat Sep 03, 2016 3:42 am
That's a huge question but to start with I suggest you spend sometime thinking about exactly what you to get out of your data and the parameters of the 'questions' you'd like to ask it. That will go a long way towards helping you to decide how to design your database.
Do you need second by second information? Will you be concerned by field sizes or distances? Per-play or in-play? Jockeys & trainers? Do you need to know who won? That sort of thing.
My personal approach is to capture and store anything and everything I can in the finest possible level of detail possible. That leaves you with a huge unwieldy set of data almost useless on it's own....BUT then I use that 'master' data to create a seperate query database just containing specific things pertinant to that query, pre-sorted in the appropriate way, containing no irrelevant data, with additional items derived from ranges of data from the 'master' data....then I run my enquiries and reports against that data.
For instance, I might want to know the price at two points in time, but also want to know the lowest and highest price reached. So what would I do is scan the 'master' data for the lowest/highest on each selection, and then when I pull out the 1 line per second stuff, I can put the low/high number on each line.
Then when you extract the price at say 9mins and the price at say 28seconds, you don't have to read every line to see what the min/max was inbetween.
It's tricky stuff with a lot of ways to do it, so as I said earlier...have a good long think about the outputs you want, and that will help you workout the inputs you need.
If all else fails (and this is a hugely underestimated technique)...image you were doing it the old fashioned way, with paper and pen and the BF data printed in a book..... What exactly would you do? Break the task down into the smallest imaginable steps and write down what each step involves....and then you can begin to think about how you can 'computerise' each of those steps. Even the most complex jobs are just a series of small managable steps.
good luck
Appreciate your thoughtful response Shaun! Indeed it's necessary to have a clear cut goal from the start. For now I will work with the free data. I do have a few ideas in mind about what I want to do with the data but not sure really how to query the data. ie, how would I scan the master data and output what im looking for? For example, if I wanted to find out in general when back prices are highest before the race (5hrs before race or 2 hrs or 10 hrs?) using a predetermined sample size, how would I go about doing this? What is a good program to use if any? Is R an option? What programming language should I look into? Are there any good resources that I should look in to? I only have rudimentary knowledge of coding and some good experience with excel but am willing to get more into it. Would appreciate any guidance on the matter 