How to analyse .tar file from Betfair

Football, Soccer - whatever you call it. It is the beautiful game.
Post Reply
User avatar
ZeroCounty
Posts: 15
Joined: Sun Aug 06, 2017 9:43 pm

Hello,

I have downloaded a .tar file from Betfair regarding Football data of last month. I want to analyze it to formulate a viable strategy. When i unzipped the file i could find a lot of folders leading to .bz2 files. I am new to this and hence have no idea on how to use this files.

I know there are few users like megarain who are well versed in this type of issue.

I do not have experience in coding but i can follow the instructions. I would appreciate if someone can provide me detailed assistance in relation to this.

Thank you.
User avatar
megarain
Posts: 2040
Joined: Thu May 16, 2013 1:26 pm
Contact:

Hiya,

Nice first post !!

You are right, in that I am doing a bunch of work in this area.

Let me explain my thinking, and what I hope is going to happen.

The Betfair Historical data website, allows registered users, who have paid some commission, to access three types of data :

Free event data, with time stamps of every minute.

1 sec event data, which ranges in price, but on avg is about £75 per month of archive, per sport.

Millisec data, which ranges in prices, but on avg, is maybe £150 per month, per sport.

The process to get what data you want, is a little poorly explained on the website, but, once you have experimented, its now fairly easy, to say, download, the sample data, for all games, Pakistan played at cricket, for the last 2 yrs.

This file, (any of the 3 options), comes as a tar file, which needs to be uncompressed, and then fed into a database.

I spent a bunch of money, building a tool, which automatically takes the tar file, and allows u to import it to a SQL database. From there, if u have the skills in SQL, u can refine it a bit, and then maybe export the file to excel, to further play with.

Missing from this process, is a quick and easy way, to view a graph of the data.

I am spending another bunch of money, on this .. but, the graphical viewer, has to be slightly different for each sport. At some point, maybe in the next mth, but definately within 2 mths, I will consider releasing a free version of this tool, and a paid one.

Now, if u just want to look at horse-racing, the free 1 minute data, is going to be pointless, in this viewer - so, users will have to ask themselves, how badly they want to pay betfair for the 1 sec stuff, but, for ther sports, where the event lasts 90 min+, I expect it will be pretty handy.

When/if I get to the stage of releasing etc, I will firstly contact board admin, to see if they are ok with this, and then post some screen shots, or maybe provide a video or 2.
User avatar
ZeroCounty
Posts: 15
Joined: Sun Aug 06, 2017 9:43 pm

Hi megarain,

Nice of you to reply to my post. The work you are doing is really commendable. I would like to help in any way possible may be as a tester or some thing. Please leave me a pm if i could help in any way. And please keep us posted on any new updates.

Again thank you for all your contributions, i am sure it has and will be of immense help for many people here.

Thank you
User avatar
ShaunWhite
Posts: 9731
Joined: Sat Sep 03, 2016 3:42 am

megarain wrote:
Sun Nov 19, 2017 3:08 am
I am doing a bunch of work in this area.
What version of SQL are you using?
My resolution for '18 is to get back into it. I haven't even seen it for about 10 yrs but as it used to be my bread and butter I should be able to pick it up again ...I hope, even though the look and feel of these things changes a lot and they keep giving old things fancy new names which doesn't help.
User avatar
megarain
Posts: 2040
Joined: Thu May 16, 2013 1:26 pm
Contact:

I have SQL on a Windows 2008 server.

To say I am a novice in this area .. is an understatement .. and by and large, I just pay people, to try get what I want.

This process is not quick.

I know there are some database size limitations in the free version of SQL, and cricket can have huge file sizes (a 5 day test match, with the game being open for a week before etc, is a seriously big file).

I think the Graph Viewer using Postgre SQl.
User avatar
ashking
Posts: 1
Joined: Wed Sep 26, 2018 6:41 pm

Hey Megarain,

I am really struggling with the betfair historic data files for days but I am unable to open them. Actually I am developing a strategy for cricket trading for which I really need price volume charts of the games and I saw that you decoded a way of going through that data,

So can you please guide me through the steps for viewing the data.. I just want to view price volume charts of the games.
Post Reply

Return to “Football trading”