Number Crunching After Capturing and Sorting.

[ShaunWhite]

I just need a green up formula now and I'm close to really doing some number crunching.

If anybody has got a formula handy it would save me some research time. Balance + current green up value across all selections.

GreenUp = Stake * ((Entry price - Exit price) / Exit price)

Do that for each open trade and sum them.

Commission on profit obv but multiple trades in the same market will reduce that a little as losses will reduce the gross gains on which commish is based.

[ruthlessimon]

I just need a green up formula now and I'm close to really doing some number crunching.

If anybody has got a formula handy it would save me some research time. Balance + current green up value across all selections.

The actual formulas I use are as follows (if you use it & find an error (or ways to simplify it) lemmi know btw )

LAY:
Stake: =LI20/0.95*INDEX(Calcs!A:A,MATCH(LI22,Calcs!A:A,0)-1)/(INDEX(Calcs!A:A,MATCH(LI22,Calcs!A:A,0)-1)-LI22)
P&L: =IF(LI21*(LI23-LI22)/LI23>0,LI21*(LI23-LI22)/LI23*0.95,LI21*(LI23-LI22)/LI23)

LI20: 1 tick profit target (i.e. £1)
LI21: Stake
LI22: Entry price
LI23: Hedge price
Calcs!A:A: Price list from 1000 - 1.01

BACK:
Stake: =LJ20/-0.95*INDEX(Calcs!A:A,MATCH(LJ22,Calcs!A:A,0)+1)/(INDEX(Calcs!A:A,MATCH(LJ22,Calcs!A:A,0)+1)-LJ22)
P&L: =IF(LJ21*(LJ22-LJ23)/LJ23>0,LJ21*(LJ22-LJ23)/LJ23*0.95,LJ21*(LJ22-LJ23)/LJ23)

LJ20: 1 tick profit target (i.e. £1)
LJ21: Stake
LJ22: Entry price
LJ23: Hedge price
Calcs!A:A: Price list from 1000 - 1.01

[ShaunWhite]

The questions around confidence in back tests and live results is a hardy perennial. Whether it can be reduced to a formula is doubtful imo. I think it just comes down to a range of confidence and experience.

Even a rock solid data edge may well deteriorate a little in live, how much will depend on what transactions are involved and experience would probably give you a guide as to how much. For that reason it's probably worth having a small deduction factored into your ideal world simulation.

As for confidence then it might be worth looking into some sort of simplified Kelly staking, back it off on the bad runs and ramp it up on the good ones. It might not be the best way to maximise returns but it will help you keep your sanity if a bot that appears to be struggling is only costing you a pound or two a day.

I've always liked the idea of having a R&D budget too, it's easier to handle lose a few quid trying an idea if you don't expect it back. Like paying to enter a competition, you don't expect a prize but it's great if you do, and how much you'll pay is related to how confident you are that you'll win something.

[ruthlessimon]

When I get to the stage that my first big clever bot project goes live, I want to know what exactly I’m looking at. Like a pilot in a plane, if the thing starts to go down I would like to know which buttons I need to press before it hits the ground.

Here's a "random" edge I'm looking currently looking at - in the context of the plane analogy - this is a full-blown engine malfunction at 20,000 feet (& why your question is so important)

The overall figures are the following:

Return on stake: -0.09%
Total trades: 2002

Now someone could say, yup no edge - move on.

But let's look at the equity curve



Suddenly it shows a nasty dilemma (had it been traded live)

The first 450 trades (big sample) = +3.92% RoS! (big edge)

Everything after 450 = -1.22% (perhaps enough of a -ve return to label it as an anti-edge)

Therefore, is it fair to label this as "random"?

[ruthlessimon]

Didn't Mark Douglas say a 20 trade sample was enough??

[ruthlessimon]

If in doubt use at least a year's data and graph out the cumulative p&l day by day - a picture's worth a thousand words.

It certainly is worth 1000 words; what's ya opinion on the graph above? Edge or no edge? (if a variable can be connected i.e. what changed)

[ShaunWhite]

(Si etc but esp) You'll like this, not a lot, but you'll like it.

Re: Edge persistance
Chart below is 1 strategy, 2 totally unrelated markets (diff sports actually).
You might say 3 months isn't much of a sample, it was 40,000 trades ! Forty thousand.
Pattern fitted? Nope, there are NO market or selection parameters. Just rinse and repeat.

Conclusion :
1. Ouside influences.
Maybe someone spotting the edge, implemented it on both sports, persisted with the grey sport but stopped it on the blue sport after a few weeks?
2. Just patterns in the clouds.
Everything is just a random walk and like a factal, you see patterns at all levels of zoom.

Questions : Is it worth going live on both, either or neither? What if you'd started the trial in late July, would you have persisted after a month? Should you persist with both, either or neither? The elephant in the room is that it's a wafer thin 0.2% roi.

There's only one certainty, this game drives you insane in the membrane

[ruthlessimon]

(Si etc but esp) You'll like this, not a lot, but you'll like it.

You might say 3 months isn't much of a sample, it was 40,000 trades ! Forty thousand.
The elephant in the room is that it's a wafer thin 0.2% roi.

There's only one certainty, this game drives you insane in the membrane

0.2% in 40000, pfftt

The best I've found so far was

1.0% in 5000 - if that was a top trump do I beat ya??



Also that 1% strat applies to all runners, meaning the RoS is skewed down

#vastsearcheffect !!

I think I lost my sanity months back

[ruthlessimon]

Questions : Is it worth going live on both, either or neither? What if you'd started the trial in late July, would you have persisted after a month? Should you persist with both, either or neither?

Exactly, this is why we need Peter!!

[ruthlessimon]

The best I've found so far was

1.0% in 5000

I was just looking at this again, & if it really was a genuine result - this thread potentially explains why: viewtopic.php?f=20&t=10411

I think people missed the ball totally, they should have really queried (to themselves) why is Peter so against candlesticks.

It's fascinating that smoothoperator took it as "an attack on his approach" whereas Peter has actually implied something of huge value - which could've massively improved his trading (had he just shifted his focus and been more malleable)

viewtopic.php?p=87477#p87477

[ruthlessimon]

I might be steering this thread slightly off topic - but hey, I'm also desperate to learn about proper number crunching myself!

Out plops another apparent "non-edge" strategy.



Never seen a parabola like it

[CallumPerry]

Much appreciated Shaun and Simon you lovely lovely men! Just had a very blonde moment when realising what I was asking for already had a name, as I was stumbling around for a way to explain what I was trying to do a quick google search has just returned a phrase I have seen loads but stupidly never thought to read about until now. Not knowing the term equity curve is pretty embarrassing.

Kelly staking is something that has intrigued me for a while now. May create some ‘confidence levels’ indicators and try different staking with different levels and see what happens. Also interested to explore some equity curve articles and try out some ideas this weekend after the kids at work let me go home (unfortunately it’s that way round for teachers these days).

You carry on leading this thread Simon lol honestly all of this is really helping me to learn, directing me with WHERE I need to learn and giving me new things to consider. All I know is I now need to collect a MUCH bigger sample.

Never called out for a response from the other mega pros on this forum until now but it would be fascinating to read how you lot go about handling big data sets (a few tutorials on how you organise and arrange such vast amounts would be beyond helpful also).

I must say that is quite the spectacular nose dive Simon lmao too small of a sample to be down to seasonality, too large to be the first part just riding a lack of noise/false signals/bad luck before getting hit by it all. That’s quite shocking actually thinking about it because I would have thought once a sample becomes so big you would see a very smooth incline/decline with less and less variance the more data you incorporated.

I do have a few more things I’d like to make comment on but it’s getting late, got one more day of work before the weekend and I’m shattered so… yeah. Hoping to read some more interesting thoughts soon. Night traders. Has anyone on this forum ever ended a message saying good night? I’m late, it’s tired. I’m off.

[ruthlessimon]

I must say that is quite the spectacular nose dive Simon lmao too small of a sample to be down to seasonality, too large to be the first part just riding a lack of noise/false signals/bad luck before getting hit by it all. That’s quite shocking actually thinking about it because I would have thought once a sample becomes so big you would see a very smooth incline/decline with less and less variance the more data you incorporated.

I extended it to my full data set & got this:



The sample is now humungous (7000 trades)

You're absolutely right, visually, the variance has dropped - but that turn really is super super fascinating. God I'm in for a late night tonight

[Naffman]

Are these all trading strategies? Is it even worth it at 0.2-1% when there's only so far you can scale it?

There's plenty of betting systems that I use and sure a lot of others do that make a profit as well as reduce the PC

[foxwood]

If in doubt use at least a year's data and graph out the cumulative p&l day by day - a picture's worth a thousand words.

It certainly is worth 1000 words; what's ya opinion on the graph above? Edge or no edge? (if a variable can be connected i.e. what changed)

It's a pic but no idea what it is a picture OF ?

No info on timescales, selective or all races, selection criteria, staking, etc

If it is a year's data as i suggested then there could be possible edges if you can find the discriminator(s) between the ups and the downs.

eg What is different about the first 450 - or was that the origin of a backfitted strategy that has now been extended and died with reversion to mean

Edit: BTW hope your implied PL is nett for these and allows for commission since that can have big consequences and wipe out any profit.