Elon Musk's Open AI beat world best dota 2 players

A place to discuss anything.
Post Reply
Wainwright
Posts: 108
Joined: Mon Jan 23, 2012 1:10 pm

Stratagem, the startup mentioned in a recent thread, are apparently doing precisely that;
Stratagem’s AI makes its calculations watching a standard, broadcast feed of the match. (Pro: it’s readily accessible. Con: it has to learn not to analyze the replays.) It tracks the ball and the players, identifying which team they’re on based on the color of their kits. The lines of the pitch are also highlighted, and all this data is transformed into a 2D map of the whole game. From this viewpoint, the software studies matches like an armchair general: it identifies what it thinks are goal-scoring chances, or the moments where the configuration of players looks right for someone to take a shot and score.
https://www.theverge.com/2017/7/6/15923 ... -stratagem

I don't see it as much of a threat to traditional trading though as they are training it by showing it historical games - once it gets to LVGs reign at Man Utd it will probably pack up through sheer boredom.
User avatar
Euler
Posts: 24700
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

Never confuse marketing with actual application in the market.

Stratagem feels like Centaur 2 to me.

If you have a brand new technology that nobody has, you just unleash it to the market and clean up.
Wainwright
Posts: 108
Joined: Mon Jan 23, 2012 1:10 pm

I imagine it to be something like this;

https://www.youtube.com/watch?v=ACmydtFDTGs
User avatar
workpeter
Posts: 165
Joined: Sat Jul 30, 2016 8:29 pm

It would be very challenging because unlike games there is less consistency. If I were to attempt using the same approach I would first need the ability to capture and replay a stream so the AI can replay the same race over and over using random strategies. This, of course, would need to be done across hundreds of races and indefinitely in order to keep updated with ever changing markets.
User avatar
Euler
Posts: 24700
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

I looked at training neural nets back in 2007, but it only seems to be trendy now. I'm not sure what has changed, computing power? The methodologies don't seem radically different?
User avatar
Dublin_Flyer
Posts: 688
Joined: Sat Feb 11, 2012 10:39 am

I think football-wise you'd need an incredibly huge amount of games to begin with, include international friendly games, friendly pre-season games that might end 15-0 to one side, add a large amount of these in so there's a full view of teams, games etc.
Include actual league games like Serie B in the last few games of the season, where form/players/expectations don't stand up to the lure of the €€ (possible bungs).
If you want the AI to have a prediction, you need it to have as many of the situations and outcomes as possible......and even then there might be a Grobbelaar in goal! :roll:
User avatar
marksmeets302
Posts: 527
Joined: Thu Dec 10, 2009 4:37 pm

Euler wrote:
Thu Aug 17, 2017 11:20 pm
I looked at training neural nets back in 2007, but it only seems to be trendy now. I'm not sure what has changed, computing power? The methodologies don't seem radically different?
Around that time a couple of improvements came into popularity that enabled training neural networks with a large number of layers. Also, gpus were used to do all the matrix calculations in parallel. The biggest change is probably the availability of much more data. I believe somebody said we now produce more data in a year than we did from the inception of mankind up to ten years ago.

Neural nets are presented as a magic potion but my own experience is that it is difficult to tailor them to specific needs. The standard problems like character recognition, finding the boundaries in a picture filled with x's and o's, etc that works fine. Doing anything that is useful for Mark seems a lot harder. The other family of algorithms that is used for learning to play games is actually the coolest thing I've learned since leaving university. Once I got how a computer can learn to play a game without even stating the rules or objectives it completely blew me away. Unfortunately, for any problem of decent size you need the combination of that algorithm with neural nets so I'm still a bit stuck.
DoctorEvil
Posts: 16
Joined: Wed Jun 21, 2017 9:07 am

Euler wrote:
Thu Aug 17, 2017 11:20 pm
I looked at training neural nets back in 2007, but it only seems to be trendy now. I'm not sure what has changed, computing power? The methodologies don't seem radically different?
I'm far from an expert, but I think computing power wouldn't have been a problem, at least not in 2007. there was some guy who tried to apply neural networks to make computer recognize objects on images (which is a pretty hard thing to do) and it went great and created lots of hype for NN.
User avatar
frogthegambler
Posts: 4
Joined: Mon Mar 12, 2018 8:09 pm

Neural Nets definitely can be used for advantage on Betfair.

I first started using them for to build value models for backing strategies pre-Betfair in 1998.

Since around 2011 I have used neural nets feeding into reinforcement learning algorithms to learn to trade with no human input. So far they work out in smaller markets like USA racing, Aust racing and greyhounds but no luck yet in UK horse racing win markets. After 7 years researching the robot keeps saying no bet!

Watching the robot learn to trade has helped with manual trading as well. It approaches things differently to what I would have expected.
User avatar
marksmeets302
Posts: 527
Joined: Thu Dec 10, 2009 4:37 pm

frogthegambler wrote:
Mon Mar 12, 2018 8:20 pm
Neural Nets definitely can be used for advantage on Betfair.

I first started using them for to build value models for backing strategies pre-Betfair in 1998.

Since around 2011 I have used neural nets feeding into reinforcement learning algorithms to learn to trade with no human input. So far they work out in smaller markets like USA racing, Aust racing and greyhounds but no luck yet in UK horse racing win markets. After 7 years researching the robot keeps saying no bet!

Watching the robot learn to trade has helped with manual trading as well. It approaches things differently to what I would have expected.
That's really cool! I've been dreaming about this for years, never got it to work. Would be great if one day you get an email from the system "I think I've got something for UK horse races" :-)
Are you using it in the way deepmind did the atari games?
User avatar
frogthegambler
Posts: 4
Joined: Mon Mar 12, 2018 8:09 pm

They used convnets combined with Q-learning. Played the games many times and used experience replay.
Using historical Betfair data you can go straight to experience replay.
The big difference is with an arcade game they can just use the screen image as the input and the game actions as the actions whereas with BF you have to decide the inputs and what actions you want to use. It gets very complicated very quickly.
I have experimented with convnets with synthetic 'images' to represent the inputs but at the moment the agents deployed just used more simple MLP nets.

To say NNs+RL = AI is not really true at all. There are so many parameters to tweak it requires a lot of fine tuning to get working.
User avatar
marksmeets302
Posts: 527
Joined: Thu Dec 10, 2009 4:37 pm

frogthegambler wrote:
Tue Mar 13, 2018 7:12 pm
They used convnets combined with Q-learning. Played the games many times and used experience replay.
Using historical Betfair data you can go straight to experience replay.
The big difference is with an arcade game they can just use the screen image as the input and the game actions as the actions whereas with BF you have to decide the inputs and what actions you want to use. It gets very complicated very quickly.
I have experimented with convnets with synthetic 'images' to represent the inputs but at the moment the agents deployed just used more simple MLP nets.

To say NNs+RL = AI is not really true at all. There are so many parameters to tweak it requires a lot of fine tuning to get working.
Fascinating. Reading your answer I realize I might have misunderstood what experience replay is: I thought it meant feeding old <state, action, reward> tuples to the network. The action and reward is obtained by playing the game.
The number of possible inputs and actions indeed quickly escalates. I restricted it to actions on a single horse, and an indicator whether I had a back/lay or no position. The simulator prohibited stacking positions. Also I used a fixed window for the timeseries, with best-back and size, last and best-lay and size. I only considered the first two horses. For the size of the back and lays I experimented with taking the log of the sizes, the ratio of those two etc. Lots to tweak for sure. Again, great that you cracked it!
sionascaig
Posts: 1053
Joined: Fri Nov 20, 2015 9:38 am

Euler wrote:
Wed Aug 16, 2017 4:05 pm
The logic is harder to define in a real life sport IMHO. In most of the AI applications I have seen so far, it's working to rigid boundaries that can be tested and retested. How would AI know a horse is running loose? But also that it is on a circular track leading back to the start or open countryside?

/quote]

I've been monitoring academic papers over the last few tears for anything on evolutionary algorithms operating within a financial market... Haven't found a single relevant paper....

Rather than the AI approach I thought that a valid alternative could be to create an algorithm that runs an a market where success is defined as making a profit, best performing ones are then combined / mutated and run again - keep going till you get success...

However that fact that there are no academic papers on it suggests to me that someone has proved perhaps at an meta level that this cannot be done. However I cant find a paper on that either )

It would be interesting to determine whether a solution to this problem "exists" and if it does than would at least have a bit of confidence in trying to search for it...
User avatar
frogthegambler
Posts: 4
Joined: Mon Mar 12, 2018 8:09 pm

I don't think you will find any papers showing it was possible to make profits using 'AI' for real trading.

When I wrote my masters paper on this showing neural nets with RL could be used to make money in the real world I made sure no copies were kept and it was not published anywhere.

The reason is if you find something that works you do not tell everyone else how to do or the advantage would be lost. (That is one reason you never even really hear Peter Webb or Caan Berry explain exactly how they trade in their videos.)

What is done privately behind the scenes is years ahead of what is published in the journals in AI and finance. There are some very good tools available now (theano, tensorflow etc) but you really do need to know the logic and maths behind it to make it work in environments that are not completely closed like an arcade game.
Iron
Posts: 6793
Joined: Fri Dec 11, 2009 10:51 pm

I wonder whether AI could ever give us the true odds of every horse in every race, based on evaluating the effect of trainer form, headgear, pedigree, form over this distance, form on this type of ground, draw, etc over hundreds of thousands of races.

Kind of like what some of us may have done when looking at Adrian Massey's database - http://adrianmassey.no-ip.org/web1/db3/thaccess.php - but on a much bigger, more sophisticated scale.

An AI programme became the best chess player in history after a few hours of self-teaching - http://www.bbc.co.uk/news/technology-42251535 - so I wouldn't rule it out.

Jeff
Post Reply

Return to “General discussion”