How long does it take to judge whether a system is working?

Trading is often about how to take the appropriate risk without exposing yourself to very human flaws.
Post Reply
dragontrades
Posts: 1248
Joined: Wed Oct 19, 2016 11:22 pm

What is the best way to record the data of the races e.g price traded ranges and volumes?
Are there any sheets on the forum that are available? I am terrible with excel
User avatar
Dallas
Posts: 22642
Joined: Sun Aug 09, 2015 10:57 pm
Location: Working From Home

dragontrades wrote:
Fri Jul 27, 2018 1:30 pm
What is the best way to record the data of the races e.g price traded ranges and volumes?
Are there any sheets on the forum that are available? I am terrible with excel
There are several data capture sheets in the data section
viewforum.php?f=54
foxwood
Posts: 390
Joined: Mon Jul 23, 2012 2:54 pm

Euler wrote:
Fri Jul 27, 2018 11:25 am
I've found niches in very tiny cross sections of the market that if I lumped it all together would have just averaged out and lost due to the commission.
Do you find these niches have longer term profitability or does the market spot the "hole" and/or revert to mean after a short period so you have to identify the next niche ? Or is that asking too much info :)
User avatar
Euler
Posts: 24676
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

Most of them seem to be structural in terms of the way the market operates. But everything changes, always.
User avatar
Kafkaesque
Posts: 886
Joined: Fri Oct 06, 2017 10:20 am

dragontrades wrote:
Thu Jul 26, 2018 10:56 pm
I have been testing a straight betting strategy in practice mode for a while now so a few days ago I started running it for real on small stakes.
It had a bad day today which has got me worried.
I know you need to run these things for a long, long time to get a good big enough sample but when is the right time to stop and adjust some parameters or kill of the strategy?
And how large does the sample size need to be before making a judgement?
The strike rate is around 1in10 and I am staking by book % so how many races would be enough to draw some conclusions? Is there a formula for figuring the sample size out?
Hopefully someone can point me in the right direction
In terms of sample size and drawing conclusions, I would only deem it legitimate once it’s mathmatically sound, and you need such a big sample size to do so, that it isn’t really relevant, particularly at the price range you’re operating in, where the variance is going to be huge. I’ll be posting a potential formula in a minute in response to another post here. I say potential because apart from sharing, I’d love to have the math sanity-checked.

As the next best thing to mathmatical certainty (or anywhere close to it), you’ll just have to do your best to dissect and analyse your spots as objectively as possible afterwards. With the goal being to try to determine, whether your net results – no matter if they’re positive or negative – are primarily the result of sound strategy or variance. Some things to consider: Did the market behave as you expected before and after you got in? Did the price at the off end up shorter or longer than the price you got? Did some freak occurence influence the end result? And anything else during the race which may suggest whether you were actually on to something or not pre. I’m not a horse racing trader, so that is most likely an incomplete list. How long or what sample size, before you can determine anything from that, is then imo a matter of how strong indicators you feel you get from the above, and how confident you are/were in the strategy in the first place.

I’d very much second Euler’s input. Subdivide your results in as many formats as you can possible think of. It’ll of course make the variance even greater, with the smaller sample for each subcategory, but it can very much open to specific spots where the strategy works best, among other things.
User avatar
Kafkaesque
Posts: 886
Joined: Fri Oct 06, 2017 10:20 am

marksmeets302 wrote:
Fri Jul 27, 2018 9:27 am
dragontrades wrote:
Thu Jul 26, 2018 10:56 pm
The strike rate is around 1in10 and I am staking by book % so how many races would be enough to draw some conclusions? Is there a formula for figuring the sample size out?
Hopefully someone can point me in the right direction
The required sample size depends on how sure you want to be of an edge. Start recording the returns, and every time calculate the mean and standard deviation. Excel will do this for you. You will see that with more samples the standard deviation will tend to become lower: most returns will resemble the mean and you will have relatively fewer outliers. If you are aiming to be 95% sure of an edge, that translates to 2 standard deviations. If you deduct 2 standard deviations from the mean and this is a positive number then you can be 95% sure that you have an edge (a positive expectancy). This is under the assumption your returns are normally distributed, which they usually are. Unless you consider really long time periods where the market starts to change. If you only want to be 68% sure (that's one standard deviation) you deduct only 1 standard deviation from the mean. Of course for this you need much less samples. 3 standard deviations is like 99.7% and you will need a lot more samples for this.
Note that you can never be 100% sure.
Once upon a time on this www thing, I stumbled across this spreadsheet, which relates the standard deviation to your average odds, and along with yield and sample size churns out a p-value. I’m no math whiz as such. If you, or anyone, would check that the formulaes behind this are sound, it would be much appreciated (and possibly of help to Dragontrades and others).
P-value_calculator.xlsx
You do not have the required permissions to view the files attached to this post.
User avatar
ShaunWhite
Posts: 9731
Joined: Sat Sep 03, 2016 3:42 am

Kafkaesque wrote:
Fri Jul 27, 2018 2:29 pm
I’d very much second Euler’s input. Subdivide your results in as many formats as you can possible think of. It’ll of course make the variance even greater, with the smaller sample for each subcategory, but it can very much open to specific spots where the strategy works best, among other things.
How is that different to pattern fitting? Not a critisism, just wondered.

And at a minimum the 'as many formats as posible' must surely have to actually relate to things likely to actually affect your strategy?
Sub dividing into grey or brown horses won't help much.
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

Euler wrote:
Fri Jul 27, 2018 11:25 am
I'd add that you should cut the data in a number of different ways. I've found niches in very tiny cross sections of the market that if I lumped it all together would have just averaged out and lost due to the commission.
Let's assume (when cutting the data) you found, removing markets priced @ 2.70, 2.72, 2.74 - massively improved a strategy (doubled its profitability). Is that enough to be immediately tradable? or is it important to understand why those 2.70, 2.72, 2.74 markets were destroying the original strategy in the first place?

Surely the latter is extremely important - because being aware of the cause, you'll be far far ahead of the market should something change?

Now I've deliberately cut the title, because I don't want people knowing what this is ;) But it's a P&L distribution based on where the entry took place. Notice how the biggest, most frequent losers are coming from entries @ 2,70, 2.72, 2.74 - but importantly there aren't any winners either. Why is this happening?? :D :D

Image
dragontrades
Posts: 1248
Joined: Wed Oct 19, 2016 11:22 pm

7/4 resistance?
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

dragontrades wrote:
Fri Jul 27, 2018 3:55 pm
7/4 resistance?
That was my initial thought, but no - because those 2.70s - 2.74s would've been clean through 7/4 on entry. Plus the markets @ 2.80 - 3.0, you;d expect would be the biggest losers - but they aren't :?
User avatar
ShaunWhite
Posts: 9731
Joined: Sat Sep 03, 2016 3:42 am

dragontrades wrote:
Fri Jul 27, 2018 3:55 pm
7/4 resistance?
7/4 Resistance
3/1 OneOfThoseThings
12/1 NobodyKnows
50/1 bar
User avatar
Euler
Posts: 24676
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

It's likely something that works on a Hcap won't work on a Mdn, there are many ways to cut it.
User avatar
ShaunWhite
Posts: 9731
Joined: Sat Sep 03, 2016 3:42 am

Euler wrote:
Fri Jul 27, 2018 4:27 pm
It's likely something that works on a Hcap won't work on a Mdn, there are many ways to cut it.
But you'd agree the cuts have to illustrate a cause, not just a correlation?
User avatar
ruthlessimon
Posts: 2094
Joined: Wed Mar 23, 2016 3:54 pm

Euler wrote:
Fri Jul 27, 2018 4:27 pm
It's likely something that works on a Hcap won't work on a Mdn, there are many ways to cut it.
Doesn't look like it :?

But surely there's a more "clever" way than simply brute forcing an unlimited pool of variables?

Image
User avatar
mcgoo
Posts: 898
Joined: Thu Jul 18, 2013 12:30 pm

Euler wrote:
Fri Jul 27, 2018 11:25 am
mcgoo wrote:
Fri Jul 27, 2018 1:31 am
If it helps I have run automation and seen returns of 10+% for 95 bets(punting system) over 2 weeks before seeing it all return to nothing in 4 days :( A trading system I have going now is doing 7% ROI over 32 trades (29% on exposure though) but I wouldn't be confident(after years of me trying this stuff) until I saw those returns over a much larger sample set(perhaps 1000 trades) but that's me -Not got a great maths brain :geek:
I'd add that you should cut the data in a number of different ways. I've found niches in very tiny cross sections of the market that if I lumped it all together would have just averaged out and lost due to the commission.
Good point, thanks. I lack the insight to do that meaningfully thus far, but it has got me thinking :)
Post Reply

Return to “Trading Psychology”