Thanks, do you have any live bots or are you still data gathering? Playing live usually highlights any flaws very quickly but also throws up lots of quirks you might want to exploit that probably won't show up in raw data.ruthlessimon wrote: ↑Sat Sep 15, 2018 6:48 pmThe benefit is excel isn't biased - & looks to be finding crazy cool looking stuff.
Number Crunching After Capturing and Sorting.
-
- Posts: 3140
- Joined: Sun Jan 31, 2010 8:06 pm
- ruthlessimon
- Posts: 2094
- Joined: Wed Mar 23, 2016 3:54 pm
Not at the moment, & the reason for this (as I've been looking through the data more closely in recent days), I can see the strategies excel are pumping out aren't optimised correctly (i.e. stupidly refined i.e. only trade between 3mins & 2mins remaining to post)) - & perhaps this is a bias of mine - but why should I trade something, that I know can definitely be improved (even if it is already an edge) - for the risk of variance/low returns.spreadbetting wrote: ↑Sat Sep 15, 2018 7:04 pmThanks, do you have any live bots or are you still data gathering? Playing live usually highlights any flaws very quickly but also throws up lots of quirks you might want to exploit that probably won't show up in raw data.
Although that's a rotten bias! One which we've talked about before
- ShaunWhite
- Posts: 9731
- Joined: Sat Sep 03, 2016 3:42 am
Plausability ∝ confidence in existence. It's stats, not religion.
Si, the saddest thing I saw recently was you saying you "need" Peter. Any self respecting addict needs more than one dealer in their phone to get themselves through times of drought, and when all else fails, grow your own.
I get it though really, what you need is confirmation you're on the right track and some assurance that you're not being mugged off. It's like those situations when you realise you always call your girlfriend but she never calls you.
I like the idea of weighting recent data, it's logical. 1915 is irrelevant so 2015 is on that relevenace slope somewhere.
Sb and foxwood speak a lot of sense.
Si, the saddest thing I saw recently was you saying you "need" Peter. Any self respecting addict needs more than one dealer in their phone to get themselves through times of drought, and when all else fails, grow your own.
I get it though really, what you need is confirmation you're on the right track and some assurance that you're not being mugged off. It's like those situations when you realise you always call your girlfriend but she never calls you.
I like the idea of weighting recent data, it's logical. 1915 is irrelevant so 2015 is on that relevenace slope somewhere.
Sb and foxwood speak a lot of sense.
- ShaunWhite
- Posts: 9731
- Joined: Sat Sep 03, 2016 3:42 am
If edges in data sometimes don't turn into a £ profit then it's perfectly possible that £ profits don't always show themselves much in data.
A change is as good as a rest. Invest £20 in trying an idea that's technically doable even if it looks marginal.
That seems to be what pw has said, and once you're working with live data and matching you'll be in a much better place to tune it.
A change is as good as a rest. Invest £20 in trying an idea that's technically doable even if it looks marginal.
That seems to be what pw has said, and once you're working with live data and matching you'll be in a much better place to tune it.
- ruthlessimon
- Posts: 2094
- Joined: Wed Mar 23, 2016 3:54 pm
... Says the guy who applied for the "Day with Peter" competitionShaunWhite wrote: ↑Sat Sep 15, 2018 7:32 pmSi, the saddest thing I saw recently was you saying you "need" Peter.
But yah like you say, it's about knowing if we're on the right track/advice for taking it to next level understanding
- ShaunWhite
- Posts: 9731
- Joined: Sat Sep 03, 2016 3:42 am
How else would I be able to install my keylogger?ruthlessimon wrote: ↑Sat Sep 15, 2018 8:34 pm... Says the guy who applied for the "Day with Peter" competitionShaunWhite wrote: ↑Sat Sep 15, 2018 7:32 pmSi, the saddest thing I saw recently was you saying you "need" Peter.
- ruthlessimon
- Posts: 2094
- Joined: Wed Mar 23, 2016 3:54 pm
Individual price movementsfoxwood wrote: ↑Sat Sep 15, 2018 6:59 pmTrying to get some idea of the timescale these pictures are representing. When you say 1500 "trades" do you mean markets or are these individual price movements you're latching on to ? What's the overall timescale the picture represents, is it a year, is it a week ?
The answer to that helps to answer if there may be an edge or not imho.
Let's say I enter @ 2.5, & exit @ 2.0, that equals a trade. If I re-enter @ 1.95, & exit @ 1.90, that equals a trade. In that market, I've taken two trades. Therefore, those 2 trades become 2 data points on the x-axis.
The timescale I'm currently using is 3mths. Although what's good is that Sept isn't included atm (uncleaned) - which means it can be tested on unseen data - which I plan to do come tomorrow.
Start putting some money through the markets, that will teach you more than any number of backtests. There is such a fine line in being profitable that a profitable strategy on paper can fall to pieces when put live, pre-race and inplay.ruthlessimon wrote: ↑Sat Sep 15, 2018 8:34 pm... Says the guy who applied for the "Day with Peter" competitionShaunWhite wrote: ↑Sat Sep 15, 2018 7:32 pmSi, the saddest thing I saw recently was you saying you "need" Peter.
But yah like you say, it's about knowing if we're on the right track/advice for taking it to next level understanding
https://www.betangel.com/blog/why-does- ... ou-use-it/
You have put up a few graphs but to me these are meaningless without knowing at high level what the strategy is doing, it looks like you are just overfitting.
Something I like to do is pick a variable / trigger which I believe to indicate predicted movement or the chance of wining and then graph it against the profit (variable on x axis). This quickly shows if there is a relationship and if my hypothesis has legs, if not rinse and repeat.
For example check out the graph below, I believed that a positive value should indicate that the horse is a value lay. Quickly graphing it confirmed this and that 4 as a trigger was a good place to start, I then confirmed it was valid with some out of sample data before putting it into action. This was only last week so I haven’t reviewed the results yet (normally leave it at least a month) but any bets placed will be far more valuable than any of the backtesting.
You do not have the required permissions to view the files attached to this post.
- ruthlessimon
- Posts: 2094
- Joined: Wed Mar 23, 2016 3:54 pm
Oh yes, I'm absolutely aware they're overfit - hence why I'm extremely cautious, & also hope to be a good example for some people that a solid looking equity curve cannot be called an edge.LinusP wrote: ↑Sat Sep 15, 2018 9:05 pmYou have put up a few graphs but to me these are meaningless without knowing at high level what the strategy is doing, it looks like you are just overfitting.
Something I like to do is pick a variable / trigger which I believe to indicate predicted movement or the chance of wining and then graph it against the profit (variable on x axis). This quickly shows if there is a relationship and if my hypothesis has legs, if not rinse and repeat.
But I'm coming at the problem from the perspective of an early Peter.
For example, if I don't know what a Maiden is, I can't build a hypothesis around how "I think" a Maiden should behave. Excel doesn't know what a novice stks race is - but if novice races generally contain "non-random" characteristics, Excel should find it. i.e. Momemtum strategies outperform on weak Novice races
This is what I believe my graphs show (but in a different (unknown) context) - which I'm trying to find out etc.
It's difficult to explain but I'd liken it to the following real physics problem.
Our current technology detected our galaxy isn't moving like it should. Something, is causing this anomaly - but we don't know what. Discover the what, we have real knowledge (https://en.wikipedia.org/wiki/Great_Attractor)
In the context of pre-race, Excel has detected an equity curve that isn't behaving like a random edge. Something is causing this anomaly - but we don't know what. Discover the what, we have real knowledge, & potentially a real edge.
The difference between the Great Attractor, & the pre-race example - Peter actually cracked it
Certainly a ramble that, apologies lads
-
- Posts: 575
- Joined: Wed Apr 19, 2017 5:12 pm
- Location: Wolverhampton
I think this thread, if nothing else, just shows how everyone approaches their research in different ways. I feel confident with what I'm doing behind the scenes and I think that in itself is the beginning of finding an edge. I've done plenty of reading (and still do each day) but I'm starting to take that information and do something with it that hopefully nobody else is. It's not revolutionary and some others may be doing (have been the last 10+ years) exactly the same but honestly I'm enjoying myself so what's the problem. I come home from work and do a bit of researching, tweaking spreadsheets, coming up with ideas etc and I compare it to how others sit down and play sudoku for a few hours. It's not about the money, it's become a Maths based hobby that hopefully one day will change my life and allow me to do wonderful things.
I feel like I'm at the stage where I'm close and that's exciting but also, I've covered all the basics. There's not much intermediate/advanced betting exchange information that's relevant for what I'm trying to do that is easily accessible on the internet (hopefully that means I'm not developing an off the shelf type of system). So it's hard to escape the beginner band but reading what some of the successful traders on this forum divulge really helps! Please keep sharing any useful tips for number crunching, interpreting, developing and deploying. It's helping!
I feel like I'm at the stage where I'm close and that's exciting but also, I've covered all the basics. There's not much intermediate/advanced betting exchange information that's relevant for what I'm trying to do that is easily accessible on the internet (hopefully that means I'm not developing an off the shelf type of system). So it's hard to escape the beginner band but reading what some of the successful traders on this forum divulge really helps! Please keep sharing any useful tips for number crunching, interpreting, developing and deploying. It's helping!
- ruthlessimon
- Posts: 2094
- Joined: Wed Mar 23, 2016 3:54 pm
I've gotta try & find the vid, but I did note down the trader's exact words:CallumPerry wrote: ↑Sun Sep 16, 2018 6:56 pmI'm starting to take that information and do something with it that hopefully nobody else is.
"The edge we have, & the reason I've been trading 30 yrs, is because everybody else thinks they know what they're doing"
It's a bold statement - but I think certainly can't be ruled out for explaining why some edges simply don't erode (although it's a bit of a paradox!!)
Linus gave you a practical example of how he identifies and explores a strategy based on analysis of the data.
Have you tried doing something similar and put money in the market to understand how theory and reality sometimes blend and other times conflict ?
Do that, then analyse the results and think about why some worked and others didn't.
Either then adjust and try again with a modified version or abandon that idea and move on to the next one.
Simples - perpetual rinse and repeat of ideas.
Have you tried doing something similar and put money in the market to understand how theory and reality sometimes blend and other times conflict ?
Do that, then analyse the results and think about why some worked and others didn't.
Either then adjust and try again with a modified version or abandon that idea and move on to the next one.
Simples - perpetual rinse and repeat of ideas.
- ruthlessimon
- Posts: 2094
- Joined: Wed Mar 23, 2016 3:54 pm
Absolutely yus, I'll keep ya posted as the days go by - hopefully, it'll serve as a great case study for all
- ruthlessimon
- Posts: 2094
- Joined: Wed Mar 23, 2016 3:54 pm
Anyone remember doing transformations in maths??
90degree rotation?
90degree rotation?
- ruthlessimon
- Posts: 2094
- Joined: Wed Mar 23, 2016 3:54 pm
I think incorporating volume into any strategy is absolutely worth doing - the problem is - when we do - it becomes almost a multidimensional problem.
For example, how do we define "low volume". Here are a couple of factors I can think of:
1. Low volume in winter, is different to low volume in summer
2. Low volume @ 5mins, is different to low volume @ 1mins
3. Low volume @ 1.50, is different to low volume @ 15
4. Low volume @ Lingers, is different to low volume @ Chelt
5. Does low lay volume, behave differently to low back volume
What an effin nightmare
A traditional hypothesis would be low volume suggests lack of conviction - suggesting a propensity for mean-reversion - problem is my gut says it's not quite that straightforward when it comes to pre-race (i.e. that would bias mean-reversion to occurring before the liveshow - considering volume is far lower hrs prior)
Perhaps the best way to gauge the importance of each variable - is to simply build a strategy around each concept (i.e. backfitted to time, backfitted to course, backfitted to price), then see how they compare over a week/month of forward trading
For example, how do we define "low volume". Here are a couple of factors I can think of:
1. Low volume in winter, is different to low volume in summer
2. Low volume @ 5mins, is different to low volume @ 1mins
3. Low volume @ 1.50, is different to low volume @ 15
4. Low volume @ Lingers, is different to low volume @ Chelt
5. Does low lay volume, behave differently to low back volume
What an effin nightmare
A traditional hypothesis would be low volume suggests lack of conviction - suggesting a propensity for mean-reversion - problem is my gut says it's not quite that straightforward when it comes to pre-race (i.e. that would bias mean-reversion to occurring before the liveshow - considering volume is far lower hrs prior)
Perhaps the best way to gauge the importance of each variable - is to simply build a strategy around each concept (i.e. backfitted to time, backfitted to course, backfitted to price), then see how they compare over a week/month of forward trading