Where to Start? (Collecting My Own Data)

[lm1994]

Hi All,

Over the last few months, I have been focusing on developing my knowledge with automation. I am now looking to expand on this further by collecting my own data in excel from the markets that I trade which I feel will help improve certain aspects of my bots.

However, I am struggling to come across any good tutorials or explainer guides which provide insight into how to get started with all this.

I connected an old data collection sheet between Bet Angel and Excel (which was uploaded onto the forum years ago) in the hopes of trying to reverse engineer the sheet to try and figure out how the data was being taken from Bet Angel and copied into excel, which I wasn’t able to figure out.

From what I understand, collecting data is achieved by using VBA in excel (which I have a basic level of understanding with).
However, I’m not sure how I would get the sheet to pull various bits of data from Bet Angel such as my stored values etc and copy that data into excel so I can analyse it.

This seems like it will be a heck of a lot easier than manually looking over trading recordings and trying to collect data manually lol.

Any help or pointers is much appreciated.
Thanks

[Dallas]

Exactly what data you wanted to collect?

If everything you need can be done via stored values you could just export that as a .csv file after each each market as a rather than connecting a data capture spreadsheet

[chri2510]

I do it via stored values, just as Dallas said. Then, I import them in excel using a VBA script.

I don't think it's a good idea to overload yourself with loads of data when you start.

[ShaunWhite]

I don't think it's a good idea to overload yourself with loads of data when you start.

Collect any and all data. Otherwise in a year's time you'll want to to look at something you weren't collecting.

What usually happens is people spend ages collecting some sort of data in Excel and then in 6 months they find out it's not much use. It's much better imo to spend those months figuring out how to capture the raw api feed to a database (excel isn't up to the job) and then you've got everything across all sports, and much of it isn't exposed by BA anyway.

If you want to take that further in the future then you can use that data to feed a trading simulator for backtesting too, so you can test literally 00s of strategies a week if you want to. It's what the professional automation community do as a matter of course, and you'll end up with the same millisecond data Betfair charge £200/month/sport for.

If you're going to do a job do it properly because everything else is a waste of time and effort.

[decomez6]

If you're going to do a job do it properly because everything else is a waste of time and effort.

Job 1 :
-collecting accurate data
-maintaining the integrity of a database
job 2 :
- how to get a result from data manipulation ( without fitting )
job 3 :
-learning how to trade or bet .(armature)
job 4 :
using the knowledge to trade in real market conditions ( proffesional or hooby)
job 5 :
-adjustments and consistency.


OR .

you could just have fun .

- sit back watch the market .(record the screen if you have to )- ... as mentioned in opening post

- learn some basic BA automation ( some of the servants and other files were made with a strategy in mind )

-and also, every condition or stored value in BA has a strategic value behind it , ask yourself why is it there.

ASK ?
who or What are the market participants .
when are they likely active .
Humans are prone to emotional attachments and BOTS are made by humans.

BOTS fail all the time , as a matter of fact there are traders who specialize in tricking the bots to trigger .
---your opportunity is to spot this and ride along

you will gradually see a picture emerge ! opportunities are everywhere ! ( TPD included)

TAKE one simple strategy and be the MASTER of simplicity ... simple Data collection becomes a secondary outcome But not the primary objective.