Number Crunching After Capturing and Sorting.

[ruthlessimon]

I might be steering this thread slightly off topic - but hey, I'm also desperate to learn about proper number crunching myself!

Out plops another apparent "non-edge" strategy.



Never seen a parabola like it

[CallumPerry]

Much appreciated Shaun and Simon you lovely lovely men! Just had a very blonde moment when realising what I was asking for already had a name, as I was stumbling around for a way to explain what I was trying to do a quick google search has just returned a phrase I have seen loads but stupidly never thought to read about until now. Not knowing the term equity curve is pretty embarrassing.

Kelly staking is something that has intrigued me for a while now. May create some ‘confidence levels’ indicators and try different staking with different levels and see what happens. Also interested to explore some equity curve articles and try out some ideas this weekend after the kids at work let me go home (unfortunately it’s that way round for teachers these days).

You carry on leading this thread Simon lol honestly all of this is really helping me to learn, directing me with WHERE I need to learn and giving me new things to consider. All I know is I now need to collect a MUCH bigger sample.

Never called out for a response from the other mega pros on this forum until now but it would be fascinating to read how you lot go about handling big data sets (a few tutorials on how you organise and arrange such vast amounts would be beyond helpful also).

I must say that is quite the spectacular nose dive Simon lmao too small of a sample to be down to seasonality, too large to be the first part just riding a lack of noise/false signals/bad luck before getting hit by it all. That’s quite shocking actually thinking about it because I would have thought once a sample becomes so big you would see a very smooth incline/decline with less and less variance the more data you incorporated.

I do have a few more things I’d like to make comment on but it’s getting late, got one more day of work before the weekend and I’m shattered so… yeah. Hoping to read some more interesting thoughts soon. Night traders. Has anyone on this forum ever ended a message saying good night? I’m late, it’s tired. I’m off.

[ruthlessimon]

I must say that is quite the spectacular nose dive Simon lmao too small of a sample to be down to seasonality, too large to be the first part just riding a lack of noise/false signals/bad luck before getting hit by it all. That’s quite shocking actually thinking about it because I would have thought once a sample becomes so big you would see a very smooth incline/decline with less and less variance the more data you incorporated.

I extended it to my full data set & got this:



The sample is now humungous (7000 trades)

You're absolutely right, visually, the variance has dropped - but that turn really is super super fascinating. God I'm in for a late night tonight

[Naffman]

Are these all trading strategies? Is it even worth it at 0.2-1% when there's only so far you can scale it?

There's plenty of betting systems that I use and sure a lot of others do that make a profit as well as reduce the PC

[foxwood]

If in doubt use at least a year's data and graph out the cumulative p&l day by day - a picture's worth a thousand words.

It certainly is worth 1000 words; what's ya opinion on the graph above? Edge or no edge? (if a variable can be connected i.e. what changed)

It's a pic but no idea what it is a picture OF ?

No info on timescales, selective or all races, selection criteria, staking, etc

If it is a year's data as i suggested then there could be possible edges if you can find the discriminator(s) between the ups and the downs.

eg What is different about the first 450 - or was that the origin of a backfitted strategy that has now been extended and died with reversion to mean

Edit: BTW hope your implied PL is nett for these and allows for commission since that can have big consequences and wipe out any profit.

[gstar1975]

I must say that is quite the spectacular nose dive Simon lmao too small of a sample to be down to seasonality, too large to be the first part just riding a lack of noise/false signals/bad luck before getting hit by it all. That’s quite shocking actually thinking about it because I would have thought once a sample becomes so big you would see a very smooth incline/decline with less and less variance the more data you incorporated.

I extended it to my full data set & got this:



The sample is now humungous (7000 trades)

You're absolutely right, visually, the variance has dropped - but that turn really is super super fascinating. God I'm in for a late night tonight

You could smooth it out more just by putting in daily profit/loss.

[Euler]

Questions : Is it worth going live on both, either or neither? What if you'd started the trial in late July, would you have persisted after a month? Should you persist with both, either or neither?

Exactly, this is why we need Peter!!

I just bumped into this thread so keep it up here and I'll comment which some experience of what I've found works and doesn't work.

[CallumPerry]

I extended it to my full data set & got this:



The sample is now humungous (7000 trades)

[/quote]

That is much smoother but at the same time, last time it touched -1000 it reversed and flew up past 0, I wonder what will happen now the sample has just touched +1000 on the y axis

It's a pic but no idea what it is a picture OF ?

No info on timescales, selective or all races, selection criteria, staking, etc

All very correct! Leading the thread back to it’s original purpose, now you have 7,000+ sample of a hypothetical equity curve. How do you go about looking for discriminators with such a large data series?

Do traders have tips on how they sort, filter, analyse such massive amounts of numbers. It obviously depends what your variables are a lot of the time but how do you tweak your ideas based on past data without over fitting? Do you consider how scalable strategies are at this stage, is there a way to test for this? Basically what thoughts are going through your head?

I just think it would be interesting to read how the pros interpret these squiggly lines. Like Shaun asked, would you go live with both, either or neither.

If you would take a suggestion for a video idea Peter, I would personally love to see how you look at large data that you have collected. How you analyse it, what conclusions you draw from it and how you could implement an idea in the real market based on what squiggly lines on a chart are showing. Even discussing something that's not an edge would be worthwhile watching i.e. One second WOM data on the fav. What if you blindly followed a 5 second moving average and Backed whenever it hit a certain point and Layed when it hit another. Not taking into account anything else. Yours (and other's) insights on analysis would be a great watch. (Take the camera around and ask other traders on the spot what they make of something, watch them squirm for our enjoyment).

[CallumPerry]

The most recent large sample analysis I remember was from a few months back with the World Cup, let me see if I can find the video...

https://www.youtube.com/watch?v=gbWSgwTcHVs&t=794s

Now this was a fascinating watch! Would love this level of detail with analysing findings in large sample in day to day markets like horse racing.

[ruthlessimon]

Are these all trading strategies? Is it even worth it at 0.2-1% when there's only so far you can scale it?

If I don't use return on stake, I can jack up the profitability by just assuming I can get more money filled at my entry

[ruthlessimon]

there could be possible edges if you can find the discriminator(s) between the ups and the downs.

And how would you suggest I look for that? (i.e. discriminators) - this is what me & Cal have been itching to understand

Or was that the origin of a backfitted strategy that has now been extended and died with reversion to mean

So 450 swing trades (averaging +3.92% RoS) isn't enough of a confirmation?

But the definition of the death of a strategy would be a (roughly) 0% return on stake (slightly more negative due to commission; maybe -0.4%). Yet after 450 trades it's losing -1.22% on average per trade.

Edit: BTW hope your implied PL is nett for these and allows for commission since that can have big consequences and wipe out any profit.

Technically speaking, because the strategy mentioned is taking multiple trades a market at times. Commission has actually been unfairly overemphasized in the data

[ruthlessimon]

That is much smoother but at the same time, last time it touched -1000 it reversed and flew up past 0, I wonder what will happen now the sample has just touched +1000 on the y axis

I look forward to seeing that too

Btw this could be a factor for understanding what's occurring. But I'm not sure how to interpret the following info:

The overall P&L: £1,027
The overall return on stake: -0.05%

(commission on winners has been factored)

I've always assumed a negative return on stake will lead to a negative p&l - but this hasn't occurred





Maybe XO's & the entry prices are a vital factor

[foxwood]

there could be possible edges if you can find the discriminator(s) between the ups and the downs.

And how would you suggest I look for that? (i.e. discriminators) - this is what me & Cal have been itching to understand

Assuming you have the data in Excel then use a pivot table and apply filters - I'm also assuming that each data point has (or can be tied to) properties like market, race type, distance, value, class, favourite, etc ? If you have nothing else than prices at a point in time then your only reference point is probably time to official off ?

Or was that the origin of a backfitted strategy that has now been extended and died with reversion to mean

So 450 swing trades (averaging +3.92% RoS) isn't enough of a confirmation?

But the definition of the death of a strategy would be a (roughly) 0% return on stake (slightly more negative due to commission; maybe -0.4%). Yet after 450 trades it's losing -1.22% on average per trade.

How is it losing per trade - I assume your graph (p2 of the thread) is showing PL on the y axis which indicates a profit of ~400 after 450 trades.

As I say - not sure what it is a picture OF

[ruthlessimon]

Assuming you have the data in Excel then use a pivot table and apply filters - I'm also assuming that each data point has (or can be tied to) properties like market, race type, distance, value, class, favourite, etc ? If you have nothing else than prices at a point in time then your only reference point is probably time to official off ?

That's a good point, certainly doable - watch this space

How is it losing per trade

You've confused me now What do you mean by this?

I assume your graph (p2 of the thread) is showing PL on the y axis which indicates a profit of ~400 after 450 trades.

As I say - not sure what it is a picture OF

Yeah it's showing p&l on the y-axis (equity curve)

But like I said to Naff, the p&l in effect is meaningless. Raising the stakes will naturally act as leverage to any p&l (assuming those stakes can get filled @ market). But 4% is a pretty large automatable RoS (for 450 trades) when compared to other strategies.

Conventional trading literature suggests this would have been more than enough of a sample to confirm an edge - therefore I believe my example is a great case study to budding quant/discretionary traders - & why understanding/connecting the vital variables is so important

[ruthlessimon]

A problem I'm noticing will I'm doing data crunching like this, is I'm losing track of the strategy input variables. Ideally, from now, I need to keep rigorous track - to save going round & round in circles!

A capitulation after 1500 trades - again traditional literature will suggest that has to be an edge given that sample - clearly not!