Number Crunching After Capturing and Sorting.

[ruthlessimon]

If in doubt use at least a year's data and graph out the cumulative p&l day by day - a picture's worth a thousand words.

It certainly is worth 1000 words; what's ya opinion on the graph above? Edge or no edge? (if a variable can be connected i.e. what changed)

[ShaunWhite]

(Si etc but esp) You'll like this, not a lot, but you'll like it.

Re: Edge persistance
Chart below is 1 strategy, 2 totally unrelated markets (diff sports actually).
You might say 3 months isn't much of a sample, it was 40,000 trades ! Forty thousand.
Pattern fitted? Nope, there are NO market or selection parameters. Just rinse and repeat.

Conclusion :
1. Ouside influences.
Maybe someone spotting the edge, implemented it on both sports, persisted with the grey sport but stopped it on the blue sport after a few weeks?
2. Just patterns in the clouds.
Everything is just a random walk and like a factal, you see patterns at all levels of zoom.

Questions : Is it worth going live on both, either or neither? What if you'd started the trial in late July, would you have persisted after a month? Should you persist with both, either or neither? The elephant in the room is that it's a wafer thin 0.2% roi.

There's only one certainty, this game drives you insane in the membrane

[ruthlessimon]

(Si etc but esp) You'll like this, not a lot, but you'll like it.

You might say 3 months isn't much of a sample, it was 40,000 trades ! Forty thousand.
The elephant in the room is that it's a wafer thin 0.2% roi.

There's only one certainty, this game drives you insane in the membrane

0.2% in 40000, pfftt

The best I've found so far was

1.0% in 5000 - if that was a top trump do I beat ya??



Also that 1% strat applies to all runners, meaning the RoS is skewed down

#vastsearcheffect !!

I think I lost my sanity months back

[ruthlessimon]

Questions : Is it worth going live on both, either or neither? What if you'd started the trial in late July, would you have persisted after a month? Should you persist with both, either or neither?

Exactly, this is why we need Peter!!

[ruthlessimon]

The best I've found so far was

1.0% in 5000

I was just looking at this again, & if it really was a genuine result - this thread potentially explains why: viewtopic.php?f=20&t=10411

I think people missed the ball totally, they should have really queried (to themselves) why is Peter so against candlesticks.

It's fascinating that smoothoperator took it as "an attack on his approach" whereas Peter has actually implied something of huge value - which could've massively improved his trading (had he just shifted his focus and been more malleable)

viewtopic.php?p=87477#p87477

[ruthlessimon]

I might be steering this thread slightly off topic - but hey, I'm also desperate to learn about proper number crunching myself!

Out plops another apparent "non-edge" strategy.



Never seen a parabola like it

[CallumPerry]

Much appreciated Shaun and Simon you lovely lovely men! Just had a very blonde moment when realising what I was asking for already had a name, as I was stumbling around for a way to explain what I was trying to do a quick google search has just returned a phrase I have seen loads but stupidly never thought to read about until now. Not knowing the term equity curve is pretty embarrassing.

Kelly staking is something that has intrigued me for a while now. May create some ‘confidence levels’ indicators and try different staking with different levels and see what happens. Also interested to explore some equity curve articles and try out some ideas this weekend after the kids at work let me go home (unfortunately it’s that way round for teachers these days).

You carry on leading this thread Simon lol honestly all of this is really helping me to learn, directing me with WHERE I need to learn and giving me new things to consider. All I know is I now need to collect a MUCH bigger sample.

Never called out for a response from the other mega pros on this forum until now but it would be fascinating to read how you lot go about handling big data sets (a few tutorials on how you organise and arrange such vast amounts would be beyond helpful also).

I must say that is quite the spectacular nose dive Simon lmao too small of a sample to be down to seasonality, too large to be the first part just riding a lack of noise/false signals/bad luck before getting hit by it all. That’s quite shocking actually thinking about it because I would have thought once a sample becomes so big you would see a very smooth incline/decline with less and less variance the more data you incorporated.

I do have a few more things I’d like to make comment on but it’s getting late, got one more day of work before the weekend and I’m shattered so… yeah. Hoping to read some more interesting thoughts soon. Night traders. Has anyone on this forum ever ended a message saying good night? I’m late, it’s tired. I’m off.

[ruthlessimon]

I must say that is quite the spectacular nose dive Simon lmao too small of a sample to be down to seasonality, too large to be the first part just riding a lack of noise/false signals/bad luck before getting hit by it all. That’s quite shocking actually thinking about it because I would have thought once a sample becomes so big you would see a very smooth incline/decline with less and less variance the more data you incorporated.

I extended it to my full data set & got this:



The sample is now humungous (7000 trades)

You're absolutely right, visually, the variance has dropped - but that turn really is super super fascinating. God I'm in for a late night tonight

[Naffman]

Are these all trading strategies? Is it even worth it at 0.2-1% when there's only so far you can scale it?

There's plenty of betting systems that I use and sure a lot of others do that make a profit as well as reduce the PC

[foxwood]

If in doubt use at least a year's data and graph out the cumulative p&l day by day - a picture's worth a thousand words.

It certainly is worth 1000 words; what's ya opinion on the graph above? Edge or no edge? (if a variable can be connected i.e. what changed)

It's a pic but no idea what it is a picture OF ?

No info on timescales, selective or all races, selection criteria, staking, etc

If it is a year's data as i suggested then there could be possible edges if you can find the discriminator(s) between the ups and the downs.

eg What is different about the first 450 - or was that the origin of a backfitted strategy that has now been extended and died with reversion to mean

Edit: BTW hope your implied PL is nett for these and allows for commission since that can have big consequences and wipe out any profit.

[gstar1975]

I must say that is quite the spectacular nose dive Simon lmao too small of a sample to be down to seasonality, too large to be the first part just riding a lack of noise/false signals/bad luck before getting hit by it all. That’s quite shocking actually thinking about it because I would have thought once a sample becomes so big you would see a very smooth incline/decline with less and less variance the more data you incorporated.

I extended it to my full data set & got this:



The sample is now humungous (7000 trades)

You're absolutely right, visually, the variance has dropped - but that turn really is super super fascinating. God I'm in for a late night tonight

You could smooth it out more just by putting in daily profit/loss.

[Euler]

Questions : Is it worth going live on both, either or neither? What if you'd started the trial in late July, would you have persisted after a month? Should you persist with both, either or neither?

Exactly, this is why we need Peter!!

I just bumped into this thread so keep it up here and I'll comment which some experience of what I've found works and doesn't work.

[CallumPerry]

I extended it to my full data set & got this:



The sample is now humungous (7000 trades)

[/quote]

That is much smoother but at the same time, last time it touched -1000 it reversed and flew up past 0, I wonder what will happen now the sample has just touched +1000 on the y axis

It's a pic but no idea what it is a picture OF ?

No info on timescales, selective or all races, selection criteria, staking, etc

All very correct! Leading the thread back to it’s original purpose, now you have 7,000+ sample of a hypothetical equity curve. How do you go about looking for discriminators with such a large data series?

Do traders have tips on how they sort, filter, analyse such massive amounts of numbers. It obviously depends what your variables are a lot of the time but how do you tweak your ideas based on past data without over fitting? Do you consider how scalable strategies are at this stage, is there a way to test for this? Basically what thoughts are going through your head?

I just think it would be interesting to read how the pros interpret these squiggly lines. Like Shaun asked, would you go live with both, either or neither.

If you would take a suggestion for a video idea Peter, I would personally love to see how you look at large data that you have collected. How you analyse it, what conclusions you draw from it and how you could implement an idea in the real market based on what squiggly lines on a chart are showing. Even discussing something that's not an edge would be worthwhile watching i.e. One second WOM data on the fav. What if you blindly followed a 5 second moving average and Backed whenever it hit a certain point and Layed when it hit another. Not taking into account anything else. Yours (and other's) insights on analysis would be a great watch. (Take the camera around and ask other traders on the spot what they make of something, watch them squirm for our enjoyment).

[CallumPerry]

The most recent large sample analysis I remember was from a few months back with the World Cup, let me see if I can find the video...

https://www.youtube.com/watch?v=gbWSgwTcHVs&t=794s

Now this was a fascinating watch! Would love this level of detail with analysing findings in large sample in day to day markets like horse racing.

[ruthlessimon]

Are these all trading strategies? Is it even worth it at 0.2-1% when there's only so far you can scale it?

If I don't use return on stake, I can jack up the profitability by just assuming I can get more money filled at my entry