Elon Musk's Open AI beat world best dota 2 players

[frogthegambler]

They used convnets combined with Q-learning. Played the games many times and used experience replay.
Using historical Betfair data you can go straight to experience replay.
The big difference is with an arcade game they can just use the screen image as the input and the game actions as the actions whereas with BF you have to decide the inputs and what actions you want to use. It gets very complicated very quickly.
I have experimented with convnets with synthetic 'images' to represent the inputs but at the moment the agents deployed just used more simple MLP nets.

To say NNs+RL = AI is not really true at all. There are so many parameters to tweak it requires a lot of fine tuning to get working.

[marksmeets302]

They used convnets combined with Q-learning. Played the games many times and used experience replay.
Using historical Betfair data you can go straight to experience replay.
The big difference is with an arcade game they can just use the screen image as the input and the game actions as the actions whereas with BF you have to decide the inputs and what actions you want to use. It gets very complicated very quickly.
I have experimented with convnets with synthetic 'images' to represent the inputs but at the moment the agents deployed just used more simple MLP nets.

To say NNs+RL = AI is not really true at all. There are so many parameters to tweak it requires a lot of fine tuning to get working.

Fascinating. Reading your answer I realize I might have misunderstood what experience replay is: I thought it meant feeding old <state, action, reward> tuples to the network. The action and reward is obtained by playing the game.
The number of possible inputs and actions indeed quickly escalates. I restricted it to actions on a single horse, and an indicator whether I had a back/lay or no position. The simulator prohibited stacking positions. Also I used a fixed window for the timeseries, with best-back and size, last and best-lay and size. I only considered the first two horses. For the size of the back and lays I experimented with taking the log of the sizes, the ratio of those two etc. Lots to tweak for sure. Again, great that you cracked it!

[sionascaig]

The logic is harder to define in a real life sport IMHO. In most of the AI applications I have seen so far, it's working to rigid boundaries that can be tested and retested. How would AI know a horse is running loose? But also that it is on a circular track leading back to the start or open countryside?

/quote]

I've been monitoring academic papers over the last few tears for anything on evolutionary algorithms operating within a financial market... Haven't found a single relevant paper....

Rather than the AI approach I thought that a valid alternative could be to create an algorithm that runs an a market where success is defined as making a profit, best performing ones are then combined / mutated and run again - keep going till you get success...

However that fact that there are no academic papers on it suggests to me that someone has proved perhaps at an meta level that this cannot be done. However I cant find a paper on that either )

It would be interesting to determine whether a solution to this problem "exists" and if it does than would at least have a bit of confidence in trying to search for it...

[frogthegambler]

I don't think you will find any papers showing it was possible to make profits using 'AI' for real trading.

When I wrote my masters paper on this showing neural nets with RL could be used to make money in the real world I made sure no copies were kept and it was not published anywhere.

The reason is if you find something that works you do not tell everyone else how to do or the advantage would be lost. (That is one reason you never even really hear Peter Webb or Caan Berry explain exactly how they trade in their videos.)

What is done privately behind the scenes is years ahead of what is published in the journals in AI and finance. There are some very good tools available now (theano, tensorflow etc) but you really do need to know the logic and maths behind it to make it work in environments that are not completely closed like an arcade game.

[Iron]

I wonder whether AI could ever give us the true odds of every horse in every race, based on evaluating the effect of trainer form, headgear, pedigree, form over this distance, form on this type of ground, draw, etc over hundreds of thousands of races.

Kind of like what some of us may have done when looking at Adrian Massey's database - http://adrianmassey.no-ip.org/web1/db3/thaccess.php - but on a much bigger, more sophisticated scale.

An AI programme became the best chess player in history after a few hours of self-teaching - http://www.bbc.co.uk/news/technology-42251535 - so I wouldn't rule it out.

Jeff

[ShaunWhite]

I wonder whether AI could ever give us the true odds of every horse in every race, based on evaluating the effect of trainer form, headgear, pedigree, form over this distance, form on this type of ground, draw, etc over hundreds of thousands of races.

Said the CEO of every major bookmakers and every major syndicate. And it's a question that's been worked on for probably 100 yrs by the finest racing brains.

The problem is always that race horses are temperamental skittish creatures and you don't always know what they'll be like from day to day with the degree of certainty you're hoping for.

You'll never calculate true odds for an individual race and what it should have been can never be known. You'll maybe work them out over a long run but if you want that then the numbers are already there in the market.

[SeaHorseRacing]

I think intuition makes a trader and I am not so sure this could be programmed.

[frogthegambler]

I wonder whether AI could ever give us the true odds of every horse in every race, based on evaluating the effect of trainer form, headgear, pedigree, form over this distance, form on this type of ground, draw, etc over hundreds of thousands of races.

Kind of like what some of us may have done when looking at Adrian Massey's database - http://adrianmassey.no-ip.org/web1/db3/thaccess.php - but on a much bigger, more sophisticated scale.

An AI programme became the best chess player in history after a few hours of self-teaching - http://www.bbc.co.uk/news/technology-42251535 - so I wouldn't rule it out.

Jeff

A key issue with this for working out 'true' win probabilities is insider information in the market for horseracing.

Trainers and owners are allowed to back their own horses when they are fit and ready.

If you look at the price of a horse in the last 5 mins. It can go 3.1 out to 4.3 and then back to 2.8. Its true price cannot vary that much. But which price is right?

I found after about 2004 if I priced something at 3.0 and it was trading at 7.0 it merely meant my model was wrong and the market was right. Insiders knew the horse would not run to form or that another in the race would run better than its form suggested.

[cyxstudio]

I think intuition makes a trader and I am not so sure this could be programmed.

Heres the problem .

Machine learning means machines can develop intuition too after thousands of repitition.

And they are better than us at intuition.

[mcgoo]

I think intuition makes a trader and I am not so sure this could be programmed.

Heres the problem .

Machine learning means machines can develop intuition too after thousands of repitition.

And they are better than us at intuition.

The more I see of AI/ML the more I think one would be crazy not to try it on BF trading. Banks big finance must be using it-BF too.
I am learning to code at a ripe old age -just cos I like it, but I think the leap to machine learning will be a bridge too far for this brain..of course I'll keep tinkering.If nothing else it all has made me realise how far my bots have to go.What else am I gonna do with my time What a time to be alive!

[Jukebox]

It is interesting to note that while traders here ponder if computers can become more 'human' to become better trading bots there a other threads and constant swapping of info on how traders might become less 'human' to become better traders.