Why Live Results Just Doesnt Mirror Back Testing

[theuktrader]

Dear Community

I urgently need help and need to know whether anyone else is experiencing the same thing.

I have been back testing a specific football trading strategy. The back testing has gone back beyond the last 2 months and covers over 160 trades. The strategy has 66% win rate and my winners are 1.5 more than my losers. Considering that they normally say you should monitor 50-100 trades to see if something has some worth I was quite excited about going live.

When I went live the first has been nowhere what the live results were showing. The live results are showing after about 20 trades a win barely just over 50%. Whilst I am in profit because the expectations are so far down from the back test I have all of a sudden started doubting my system wondering whether the last 2 months worth of data was a fluke.

I also have a rule that I trade around 3-5 trades per day but if the first 3 trades are losers then i stop trading for that day. Should I reduce it to a maximum of 2 initial straight losers for the day to minimize my daily liability as I dont mind losing a daily battle as long as I am winning the long term war.

But just interested to know how others fare with the contrast between their backtesting results and live results.

I am aware that it doesnt have to follow back testing exactly but man, after getting a weekly win rate of 65-70% in the previous 8 weeks of back testing getting a 50% in the first week of live was difficult to understand.

theuktrader

[marksmeets302]

Backtesters are always either too optimistic or to conservative when it comes to execution. In either case it's different from reality. Every time you do a trade in your simulation you will go into a situation that slightly deviates from reality. The more you trade, the bigger this deviation will be. And then there's the market impact your trades have. Only if huge amounts are traded in the real market your own will have insignificant impact.

My advice: forget about back testing and test in the real market with small amounts. Once you are on to something, scale up until you no longer can.

[LinusP]

I also have a rule that I trade around 3-5 trades per day but if the first 3 trades are losers then i stop trading for that day. Should I reduce it to a maximum of 2 initial straight losers for the day to minimize my daily liability as I dont mind losing a daily battle as long as I am winning the long term war.

So your strategy is dependent on the probability of the first 3 being winners? Theres your problem.

[Trader Pat]

I think the answer is in your first sentence. A sample size of 160 over 2 months is not a big enough sample size, you should probably be aiming for a sample size of at least 1000 trades.

A few years ago I was playing around with over / unders in matches that met certain conditions and I found it took several hundred markets before I got a reliable set of data when all the noise and natural fluctuation was taken into account.

[theuktrader]

Thank you for all your kind advice.

No I am not depending on the first 3 trades being winners as I am happy with a starting sequence of LLW WLW LWL WLL especially considering my winners are 1.5 the size of my losers. I am just saying that if I hit LLL then thats like my stop loss for the day because I am assuming one has to have some circuit breaker for the day or you could just go on and on and ending up with LLLLLL......Dont you have some limit for the day or do you just go on and on and on trading regardless of the results. I thought as traders we were supposed to have a liability limit for the day to protect our bank and protect our pyschology???

Thanks

[spreadbetting]

If your strategy is sound all you do by stopping for the day is limit any potential profits. Traders may well give up because of a bad day but they're not following a set strategy they're more likely making decisions on the fly so their pyschology may well play a big part of their profits. For any systems people usually build in some kind of bank protection from their staking plan not by giving up after 3 results.

[ShaunWhite]

Thank you for all your kind advice.

No I am not depending on the first 3 trades being winners as I am happy with a starting sequence of LLW WLW LWL WLL especially considering my winners are 1.5 the size of my losers. I am just saying that if I hit LLL then thats like my stop loss for the day because I am assuming one has to have some circuit breaker for the day or you could just go on and on and ending up with LLLLLL......Dont you have some limit for the day or do you just go on and on and on trading regardless of the results. I thought as traders we were supposed to have a liability limit for the day to protect our bank and protect our pyschology???

Thanks

The concept of a 'day' is irrelevent, another trade on the same day is no different to a trade on the next day. When you look back at your graph of 500 matches, can you tell where each day starts and ends, no.

[ShaunWhite]

Also were you back testing against data or do you mean in practice mode? Theoretical prices in data are not possible to hit in live mode as the prices will often be taken before you take them.

[theuktrader]

That is a fantastic insight.

Had never really thought of it in that way that the day is irrelevant if looking back over a historic sample size of 500 trades.

So I presume you are trying to tell me that you simply have to take every trade that comes along which meets your criteria even if it means going through a losing streak.

Thanks for your insight

[theuktrader]

the back testing is based on match selection meeting certain criteria for a particular price range that games descend to when the match hits a certain time in the match.

[ShaunWhite]

you simply have to take every trade that comes along which meets your criteria even if it means going through a losing streak.

Streaks will happen, I remember a while ago there was a streak of about 30 losing favs on the horses (could be wrong about that). That's why escalating stakes are a no no and why it's the strike rate over many hundereds or thousands of events that matters.

There's a formula for the likelihood of streaks for a given strike rate but I can't remember it.

[xitian]

Welcome to the world of systematic betting! For a system which is more gambly, which yours is because you're not trading out and a sport like football has large discrete jumps in prices, it can be quite hard to tell if you have something which will be profitable in the long run. At least you've done the first right thing which is to question your live results early on, and not just pile in more money to recover because your backtest said it would work!

I'd agree with what everyone else has mentioned, but just to reiterate:
- Definitely not a big enough sample. For something that's gambly (thinking betting on an outcome rather than trading changes in a price) then I'd expect you to need 1000s of samples. I have a football strategy which I test over a year of data which bets on around 30,000 matches in that year. I'm /pretty/ sure it's profitable, but even then I've probably overfitted my parameters to an extent.
- First thing to check is that your live trades match your simulated system. So re-simulate over those few days that you've bet live. Does the simulation give you the same P&L as live? If it does, at least that confirms your backtest during your backtest period would at least have matched real betting. If it doesn't then it means your backtest system doesn't represent live betting close enough for whatever reason (and there could be quite a few reasons for this).
- I can see how stopping after 3 losses in a day sort of acts as a stop loss, but there are better ways to handle that. If your system works, there's no reason to stop early - you're just throwing away opportunities. Since each football match is an independent event, and you've shown with a realistic backtest that your bets are good value then a losing run of 3 should eventually be recovered. You only really need a stop loss for operational losses or if you have no faith in your system. I personally don't use stop losses.
- Once you've increased your sample size, see if your strategy is robust by tweaking your parameters. If it only works for very specific parameters then it's not likely to work long term and you've probably stumbled on it by luck or by overfitting the parameters.
- There's more I could add, but I'd just advise you search the forum for other posts about determining system profitability. There are a few.

Good luck!

[theuktrader]

Dear Xitian

Could you clarify the difference between betting and trading. You said betting is where you are making a decision on an outcome whereas trading is where you are making a decision based on price. However in both cases what decides whether you are in profit or not is what happens in the game (I am making reference here to trading inplay).

Leaving an open back or lay and allowing the event to run to expiry is no different to trading financial options where you can initiate your trade and allow it to run to expiry. However they dont call financial options gambling. They call it trading. So unsure as to why in sports trading that a trade that is allowed to go to expiry without doing the corresponding back or lay is deemed to be gambling/betting.

I thought gambling was going to your bookies and taking their odds on offer whereas trading was using a financial exchange platform like Betfair to buy/sell with other people on the exchange.

Probably just semantics but curious to know what people think as its an interesting one.

Cheers

theuktrader

[Kafkaesque]

Dear Xitian

Could you clarify the difference between betting and trading. You said betting is where you are making a decision on an outcome whereas trading is where you are making a decision based on price. However in both cases what decides whether you are in profit or not is what happens in the game (I am making reference here to trading inplay).

Leaving an open back or lay and allowing the event to run to expiry is no different to trading financial options where you can initiate your trade and allow it to run to expiry. However they dont call financial options gambling. They call it trading. So unsure as to why in sports trading that a trade that is allowed to go to expiry without doing the corresponding back or lay is deemed to be gambling/betting.

I thought gambling was going to your bookies and taking their odds on offer whereas trading was using a financial exchange platform like Betfair to buy/sell with other people on the exchange.

Probably just semantics but curious to know what people think as its an interesting one.

Cheers

theuktrader

One major difference is your bankroll management and staking. Always trading out allows for a much more aggressive staking approach. As soon as it becomes a "sometimes letting it run", the line between what would be sound staking for betting and trading becomes quite blurred, and you're likely left between a rock and a hard place. The rock being using staking that's too aggressive and gives too much variance and/or, risk. The hard place being not aggressive enough meaning not using your bankroll as efficiently as you could.

[ShaunWhite]

Could you clarify the difference between betting and trading.

viewtopic.php?f=2&t=15067&hilit=gambling+and+trading