Back testing method/software

[firech8787]

What's the best method/ software to be able to back test a horse racing strategy?

Thanks guys

[sniffer66]

What's the best method/ software to be able to back test a horse racing strategy?

Thanks guys

A trading strategy i.e B2L, or straight betting (pre off or in-play)?

[johnsheppard]

BetAngel I don't think there is any such software out there other than custom jobbies.

If you experiment with the practice mode in Betangel you will see it's very difficult to accurately test your position in queue. Beyond that, I think you'll find that most people who have gone down the backtesting route say you're better off trying to model things than backtest.

[Anbell]

Flumine is one option

[Euler]

Back testing is statistical deja vu

[lunswor]

Back testing is statistical deja vu

Just because you can't make it work doesn't mean others have not

[conduirez]

Back testing is statistical deja vu

Just because you can't make it work doesn't mean others have not

I also find Back Testing is unnecessary, and as Euler says it is deja vu.

[Euler]

Back testing is statistical deja vu

Just because you can't make it work doesn't mean others have not

Do what ever you feel is important, but I think it's flawed and the least effective way to create a winning betting or trading system. You miss some of the most important aspects of the market.

I don't know how experienced you are or whether you are using data. But just because something happened in the past it doesn't mean it will happen in the future and it's important to know why. So I model markets rather than back test.

If a horse has shortended in price by 50% on average in the last five races, that doesn't contain any information about the next race.

[firlandsfarm]

How are you defining modelling Peter. It's all based on past stats in one form or another.

[Euler]

How are you defining modelling Peter. It's all based on past stats in one form or another.

Yes, you need historic data to model.

When I hear most people talk about backtesting they ask something like: -

When this team is 1-0 up in the 30th minute how often do they win.

Or

This horse has won three times out of five when it starts at odds of 3.00

That sort of stuff, it just doesn't make sense. But I know it is very alluring to many.

[conduirez]

How are you defining modelling Peter. It's all based on past stats in one form or another.

Yes, you need historic data to model.

When I hear most people talk about backtesting they ask something like: -

When this team is 1-0 up in the 30th minute how often do they win.

Or

This horse has won three times out of five when it starts at odds of 3.00

That sort of stuff, it just doesn't make sense. But I know it is very alluring to many.

I have to say back testing is dangerous, it can give a false sense of security.

When back testing most people do not have all the information that is available in an actual live market, and in some cases will just use the SP or as I say the happy ending of the market, they do not use the actual price at the time they would enter the market, they usually do not account for the queue, their trades are made instantaneously for the full amount, there is no account for BF taking their percentage before the next market, they are prone to over or underestimate certain outcomes. Too many times I have heard people say well it made money in the back test, why does it not make money in live markets. That is only if the back testing algorithm is working without bugs, if it is working with bugs or in a positive way well you are suddenly £1million up after a 1000 events.

Statistics will give you the historic answers. If you want to test whether your bot works, then do the test for pennies or in practice mode, however do not use the bot fully until you test it on the live market for pennies, not in practice mode.

[andy28]

I get what people are saying, but here's where I'm scratching my head:

I dive into old data to find an edge. When I think I've spotted one, I test it out in practice mode and set up automation to do its thing. I also keep gathering more data during this practice run, usually for about six months, to see if it matches what my initial edge showed.

So, after crunching numbers from this six-month trial with over 2000 races, I found it didn't make a profit every week, but it did bring in profit every month, just like my original findings suggested.

So, at the start of this month, I flipped the switch to live mode, and within two days, it's doing exactly what I hoped for. It's not some fancy trading bot, just a straightforward back or lay bot.

But here's the thing bugging me, was all this just backfitting? Even after looking at over 2000 races, some folks argue that's not enough data. Did I waste 8 to 10 months on something that'll end up making zero profit?

I get that it's tough for others to weigh in without knowing the nitty-gritty of the data I'm using. Let's just say it's kinda like what Peter said about a team going up 1-0. But with this bot, I've taken it further. In Peter's scenario, I'd wanna know not just when the goal was scored and who they were playing, but also stuff like the xG of both teams and how many goals were expected in that match.

[sniffer66]

I get what people are saying, but here's where I'm scratching my head:

I dive into old data to find an edge. When I think I've spotted one, I test it out in practice mode and set up automation to do its thing. I also keep gathering more data during this practice run, usually for about six months, to see if it matches what my initial edge showed.

So, after crunching numbers from this six-month trial with over 2000 races, I found it didn't make a profit every week, but it did bring in profit every month, just like my original findings suggested.

So, at the start of this month, I flipped the switch to live mode, and within two days, it's doing exactly what I hoped for. It's not some fancy trading bot, just a straightforward back or lay bot.

But here's the thing bugging me, was all this just backfitting? Even after looking at over 2000 races, some folks argue that's not enough data. Did I waste 8 to 10 months on something that'll end up making zero profit?

I get that it's tough for others to weigh in without knowing the nitty-gritty of the data I'm using. Let's just say it's kinda like what Peter said about a team going up 1-0. But with this bot, I've taken it further. In Peter's scenario, I'd wanna know not just when the goal was scored and who they were playing, but also stuff like the xG of both teams and how many goals were expected in that match.

Sean and Jim's posts on this forum re backtesting and modelling have been very useful, and are worth a read. One of the key points is not to gauge results based on 100% of your dataset. I've a hell of a lot of data stored (about 1.6m rows, across 50 data points) and I've split that 50/50, then modelled on 80% of the first 50% to form a strategy based on results. Then I've applied that to the remaining 20% I didn't model with to see if results were the same without backfitted data
Once I was happy results were the same. I then applied the strat to the 2nd 50% I hadn't used to model with at all - again with similar results, proving that results (probably) weren't due to backfitting

Only then, am I applying that to live in play data and recording paper results to see if it translates to live.

If you fail to do the 80/20 (or similar) split on your training dataset, and test against data not seen for training, all you are doing is backfitting. I've been there before a few times and failed miserably

[Duaneyk]

I get what people are saying, but here's where I'm scratching my head:

I dive into old data to find an edge. When I think I've spotted one, I test it out in practice mode and set up automation to do its thing. I also keep gathering more data during this practice run, usually for about six months, to see if it matches what my initial edge showed.

So, after crunching numbers from this six-month trial with over 2000 races, I found it didn't make a profit every week, but it did bring in profit every month, just like my original findings suggested.

So, at the start of this month, I flipped the switch to live mode, and within two days, it's doing exactly what I hoped for. It's not some fancy trading bot, just a straightforward back or lay bot.

But here's the thing bugging me, was all this just backfitting? Even after looking at over 2000 races, some folks argue that's not enough data. Did I waste 8 to 10 months on something that'll end up making zero profit?

I get that it's tough for others to weigh in without knowing the nitty-gritty of the data I'm using. Let's just say it's kinda like what Peter said about a team going up 1-0. But with this bot, I've taken it further. In Peter's scenario, I'd wanna know not just when the goal was scored and who they were playing, but also stuff like the xG of both teams and how many goals were expected in that match.

Whether you are back fitting probably depends on how much you are filtering data. If you are filtering 10 variables to make a profit, you're back fitting.
If you have months worth of data, apply your theory to odd months only. Once you think you have something, apply it to the even months to see if it still worked, rather than as one big lump . Then you can decide what you want to do going forward.
For me personally, I need set and forgets as I work full time so can't monitor the markets like others can, so I need data to come to a certain conclusion.

[Anbell]

If you fail to do the 80/20 (or similar) split on your training dataset, and test against data not seen for training, all you are doing is backfitting. I've been there before a few times and failed miserably

And so say all of us!