Odds feeds and APIs.

[napshnap]

Guys, can somebody recommend reliable and fast odds feed or API?
I've contacted by some oddsfeed's providers, but their prices are terrible, cause they provide their services for bookies. For now I scalping few sites for odds but it's a pain in the ass with all that Java crap and constant site's design changes.

[Euler]

We've contacted a few providers but the fees are prohibitive. Ultimately the underlying sport wants to charge as must as possible for the data and that seems to the problem. Scraping is a good way forward IMHO.

[jimibt]

alas, been down this road many times and have ended up having to resort to scraping the websites themselves. as you'll have discovered, some sites (such as willhill) put javascript traps in place to prevent blind scraping. In cases such as this, you may need to do a headless browser session (using selenium or similar) and then save the cookies before proceeding with your lighter weights scrapes. i've also found that the mobile versions of many of the sites are far easier to quickly navigate and scrape. of course, as time goes on, as you've noted, site changes can make the whole process a bit of a task to maintain. there are some sites (such as stanjames etc) where you can go under the covers and grab the xml and parse the tree recursively to get to the odds, thus making site changes less of an issue.

In short, a combo of selenium and something like htmlagility pack are your friends here. i did at one time offer a restapi onto my bookie scrapes but found that even as a free service, i ended up supporting changes and requests that made it too unwieldy for a one man band. by using IOC (inversion of control), you should be able to hotswap between individual bookie scrapes and single services such as oddschecker to fulfil your needs.

good luck

[napshnap]

TXODDS just replied, they want 5 tons for horseracing and greyhounds feed .

[napshnap]

We've contacted a few providers but the fees are prohibitive. Ultimately the underlying sport wants to charge as must as possible for the data and that seems to the problem. Scraping is a good way forward IMHO.

It is but as I said, sometimes it can be a pain in the ass, it's not cool to be always alert to small, stupid design changes which can destroy your scalper and leave you without odds in a critical moment.
That's just stupid that there is a good niche, good possibility to offer good service for small players and nobody wants to offer it. There is money to be made i tell you.

Thanks jimibt.

[jimibt]

the problem, as you know, is that scraping the odds for your own use can be problematic (ip blacklisting etc). if you were to offer a service such as this that was working on the *grey* odds market, you'd be quickly approached by litigious types from the bookies which would quickly put an end to any profitable small time service... been there, done it

[napshnap]

the problem, as you know, is that scraping the odds for your own use can be problematic (ip blacklisting etc). if you were to offer a service such as this that was working on the *grey* odds market, you'd be quickly approached by litigious types from the bookies which would quickly put an end to any profitable small time service... been there, done it

Maybe network of amazon instaces with dynamic ips can solve this problem?

[jimibt]

i used a service called proxy-mesh (http://proxymesh.com/) to get round it, but that too became expensive as the service actually would be downloading the entire series of pages every 60 seconds for about 8-10 hours a day. base costs, tho not hugely expensive, worked out at about $50 a month for the 4-5 core bookies that i was targetting

[napshnap]

So here is my business model: get network with dynamic IPs (or services like jimibt suggested), hire a good site designer and few third world low cost programmers for scalping routine, get like 20-30 100pounds subscribers - finally, swim in a pile of gold and cash like fkn Scrooge mcduck))

[napshnap]

^ or buy odds feed from those greedy bustards and then simply resell it to small players. Sounds simple.

[Euler]

^ or buy odds feed from those greedy bustards and then simply resell it to small players. Sounds simple.

The agreements you have to sign prohibit resale.

[LinusP]

As mentioned analysing the mobile app is normally the easiest due to it almost always being json / REST based. AWS use elastic IP's for there ec2 instances and charge you for reassigning and leaving unassigned. But with the rise of serverless (lambda) you can get around the blacklisting as you are assigned a new IP for every execution. Lambda on AWS also has a free tier so ignoring database costs and if you were clever you could probably run a scraping company for almost nothing....

[cheesewithmanyholes]

If there is anyone who has some nicely loaded API incl live odds, PM me pls. Or in case you wanna share.

[Kallie_Monaco]

Txodds is stealing fixtures data from other sources like snooker.org, sofascore, ITF and much more...........