General discussion : Elon Musk's Open AI beat world best dota 2 players

A place to discuss anything.
User avatar
frogthegambler
Posts: 4
Joined: Mon Mar 12, 2018 8:09 pm

Tue Mar 13, 2018 7:12 pm

They used convnets combined with Q-learning. Played the games many times and used experience replay.
Using historical Betfair data you can go straight to experience replay.
The big difference is with an arcade game they can just use the screen image as the input and the game actions as the actions whereas with BF you have to decide the inputs and what actions you want to use. It gets very complicated very quickly.
I have experimented with convnets with synthetic 'images' to represent the inputs but at the moment the agents deployed just used more simple MLP nets.

To say NNs+RL = AI is not really true at all. There are so many parameters to tweak it requires a lot of fine tuning to get working.

User avatar
marksmeets302
Posts: 506
Joined: Thu Dec 10, 2009 4:37 pm

Wed Mar 14, 2018 8:13 am

frogthegambler wrote:
Tue Mar 13, 2018 7:12 pm
They used convnets combined with Q-learning. Played the games many times and used experience replay.
Using historical Betfair data you can go straight to experience replay.
The big difference is with an arcade game they can just use the screen image as the input and the game actions as the actions whereas with BF you have to decide the inputs and what actions you want to use. It gets very complicated very quickly.
I have experimented with convnets with synthetic 'images' to represent the inputs but at the moment the agents deployed just used more simple MLP nets.

To say NNs+RL = AI is not really true at all. There are so many parameters to tweak it requires a lot of fine tuning to get working.
Fascinating. Reading your answer I realize I might have misunderstood what experience replay is: I thought it meant feeding old <state, action, reward> tuples to the network. The action and reward is obtained by playing the game.
The number of possible inputs and actions indeed quickly escalates. I restricted it to actions on a single horse, and an indicator whether I had a back/lay or no position. The simulator prohibited stacking positions. Also I used a fixed window for the timeseries, with best-back and size, last and best-lay and size. I only considered the first two horses. For the size of the back and lays I experimented with taking the log of the sizes, the ratio of those two etc. Lots to tweak for sure. Again, great that you cracked it!

sionascaig
Posts: 167
Joined: Fri Nov 20, 2015 9:38 am

Wed Mar 14, 2018 10:15 am

Euler wrote:
Wed Aug 16, 2017 4:05 pm
The logic is harder to define in a real life sport IMHO. In most of the AI applications I have seen so far, it's working to rigid boundaries that can be tested and retested. How would AI know a horse is running loose? But also that it is on a circular track leading back to the start or open countryside?

/quote]

I've been monitoring academic papers over the last few tears for anything on evolutionary algorithms operating within a financial market... Haven't found a single relevant paper....

Rather than the AI approach I thought that a valid alternative could be to create an algorithm that runs an a market where success is defined as making a profit, best performing ones are then combined / mutated and run again - keep going till you get success...

However that fact that there are no academic papers on it suggests to me that someone has proved perhaps at an meta level that this cannot be done. However I cant find a paper on that either )

It would be interesting to determine whether a solution to this problem "exists" and if it does than would at least have a bit of confidence in trying to search for it...

User avatar
frogthegambler
Posts: 4
Joined: Mon Mar 12, 2018 8:09 pm

Sun Mar 18, 2018 12:18 pm

I don't think you will find any papers showing it was possible to make profits using 'AI' for real trading.

When I wrote my masters paper on this showing neural nets with RL could be used to make money in the real world I made sure no copies were kept and it was not published anywhere.

The reason is if you find something that works you do not tell everyone else how to do or the advantage would be lost. (That is one reason you never even really hear Peter Webb or Caan Berry explain exactly how they trade in their videos.)

What is done privately behind the scenes is years ahead of what is published in the journals in AI and finance. There are some very good tools available now (theano, tensorflow etc) but you really do need to know the logic and maths behind it to make it work in environments that are not completely closed like an arcade game.

Iron
Posts: 6793
Joined: Fri Dec 11, 2009 10:51 pm

Sun Mar 18, 2018 8:48 pm

I wonder whether AI could ever give us the true odds of every horse in every race, based on evaluating the effect of trainer form, headgear, pedigree, form over this distance, form on this type of ground, draw, etc over hundreds of thousands of races.

Kind of like what some of us may have done when looking at Adrian Massey's database - http://adrianmassey.no-ip.org/web1/db3/thaccess.php - but on a much bigger, more sophisticated scale.

An AI programme became the best chess player in history after a few hours of self-teaching - http://www.bbc.co.uk/news/technology-42251535 - so I wouldn't rule it out.

Jeff

User avatar
ShaunWhite
Posts: 3793
Joined: Sat Sep 03, 2016 3:42 am

Sun Mar 18, 2018 9:06 pm

Ferru123 wrote:
Sun Mar 18, 2018 8:48 pm
I wonder whether AI could ever give us the true odds of every horse in every race, based on evaluating the effect of trainer form, headgear, pedigree, form over this distance, form on this type of ground, draw, etc over hundreds of thousands of races.
Said the CEO of every major bookmakers and every major syndicate. And it's a question that's been worked on for probably 100 yrs by the finest racing brains.

The problem is always that race horses are temperamental skittish creatures and you don't always know what they'll be like from day to day with the degree of certainty you're hoping for.

You'll never calculate true odds for an individual race and what it should have been can never be known. You'll maybe work them out over a long run but if you want that then the numbers are already there in the market.

User avatar
SeaHorseRacing
Posts: 2805
Joined: Fri May 20, 2016 7:06 pm

Sun Mar 18, 2018 9:39 pm

I think intuition makes a trader and I am not so sure this could be programmed.

User avatar
frogthegambler
Posts: 4
Joined: Mon Mar 12, 2018 8:09 pm

Mon Mar 19, 2018 7:57 am

Ferru123 wrote:
Sun Mar 18, 2018 8:48 pm
I wonder whether AI could ever give us the true odds of every horse in every race, based on evaluating the effect of trainer form, headgear, pedigree, form over this distance, form on this type of ground, draw, etc over hundreds of thousands of races.

Kind of like what some of us may have done when looking at Adrian Massey's database - http://adrianmassey.no-ip.org/web1/db3/thaccess.php - but on a much bigger, more sophisticated scale.

An AI programme became the best chess player in history after a few hours of self-teaching - http://www.bbc.co.uk/news/technology-42251535 - so I wouldn't rule it out.

Jeff
A key issue with this for working out 'true' win probabilities is insider information in the market for horseracing.

Trainers and owners are allowed to back their own horses when they are fit and ready.

If you look at the price of a horse in the last 5 mins. It can go 3.1 out to 4.3 and then back to 2.8. Its true price cannot vary that much. But which price is right?

I found after about 2004 if I priced something at 3.0 and it was trading at 7.0 it merely meant my model was wrong and the market was right. Insiders knew the horse would not run to form or that another in the race would run better than its form suggested.

cyxstudio
Posts: 256
Joined: Sun Mar 23, 2014 11:18 pm

Fri Mar 23, 2018 9:33 am

SeaHorseRacing wrote:
Sun Mar 18, 2018 9:39 pm
I think intuition makes a trader and I am not so sure this could be programmed.
Heres the problem .

Machine learning means machines can develop intuition too after thousands of repitition.

And they are better than us at intuition.

User avatar
mcgoo
Posts: 385
Joined: Thu Jul 18, 2013 12:30 pm

Sun Sep 02, 2018 1:33 am

cyxstudio wrote:
Fri Mar 23, 2018 9:33 am
SeaHorseRacing wrote:
Sun Mar 18, 2018 9:39 pm
I think intuition makes a trader and I am not so sure this could be programmed.
Heres the problem .

Machine learning means machines can develop intuition too after thousands of repitition.

And they are better than us at intuition.
The more I see of AI/ML the more I think one would be crazy not to try it on BF trading. Banks big finance must be using it-BF too.
I am learning to code at a ripe old age -just cos I like it, but I think the leap to machine learning will be a bridge too far for this brain..of course I'll keep tinkering.If nothing else it all has made me realise how far my bots have to go.What else am I gonna do with my time :D What a time to be alive! :ugeek:

Post Reply

Return to “General discussion”

  • Information
  • Who is online

    Users browsing this forum: Bing [Bot], Exabot [Bot] and 3 guests