Programming/Coding on Betfair

[johnsheppard]

Hello there,

This post may seem a bit disjointed...in a rush...

My experience is with C#. I can tell you I spend more time learning to program than I do learning to trade I still have a loooong way to go to be a good trader....So it depends on what you want to do...I am sorta almost their in terms of being a half decent software engineer (but le me stress the half decent part) and I've been tinkering with programming for 15 years and have a computing science degree (didnt learn too much there tho)....

It's not just the syntax you have to learn...that stuffs easy...almost any idiot can do it.....its the architecture and design practices that you get caught on that cost you....and nothing will teach you that except long hard experience born from mistakes....

If you are deciding on a language based on UI, you should also question whether you really need a user interface (other than a textual console that is)... as soon as you need a user interface in C#/WPF you typically then have to learn things like MVVM (which is very confusing for the uninitiated)....just like trading, every software guy on the planet has a different idea of how to do things....and that means....you need to know every different way of doing things

Look into things like Domain Driven Design, Onion Architecture/Clean Architecture...etc...there are 600 page books on the subject and they don't learn cheap....

But it depends on how hardcore you want to get, and how side tracked your personality gets you....you WILL spend a lifetime becoming an experienced software engineer...

But its good fun...so I heartily recommend it

[LinusP]

In my option in python is the best language to learn as a beginner, there are so many brilliant open source libraries available, requests / pandas / scipy / Django. It allows you to do a lot but with minimal experience, this of course comes at a cost due to it being a high level language it is never going to be as quick as C.

I was lucky that I started learning when the new API was being rolled out, I had 2/3 strategies running on BA that I migrated across to python in a month (from zero programming experience) I then soon realized that the open source wrappers available were a load of crap and not maintained so wrote my own. I have then just kept on top of changes and was one of the first to develop streaming which opened up a lot of doors!

In regards to GUI’s this is not what python is for, tkinter will make you want to kill yourself. Instead you should look at building a web app on top of an API or database, this is my setup:

Web App (vueJS / d3 / highcharts / redux, this gives me a view on profit per market / strategy / errors / uptime etc.)

Trading Instances (multiple servers running multiple strategies all in docker containers, managed with portainer)

API (sits on top of db, never connect directly to a production db without an API!)

Transaction Database (all order / strategy / market / metadata)

CloudWatch (logs from instances / db / api / docker)

S3 (recorded raw streaming data, I have a lot!)

The above is missing a data lake which sits separately that receives all data from the transaction db and all logs which are joined together to allow detailed analysis, jupyter then sits on top for further analysis.

This is then all replicated in a dev env, terraform is a lifesaver, I then use serverless to handle restarts / daily scripts. Average AWS bill is around £250-500 a month depending on what I have been doing but the majority of that is from db costs. Hopefully this won’t put people off as you can start with the trading framework like I did and then slowly build out, this is 6/7 years of hard work

This is my web app which I use to monitor things whilst lying on the beach

55740446-C6B3-4AF1-9A2E-D008C8A869BD.png

[northbound]

LinusP, your web app looks great, nice job. But I'm sure you scared people away with that £250+ monthly bill.

So here's a little message to reassure beginners that you can still accomplish a lot with a tiny $5/month Linux server that runs your Python or Nodejs app.

You wouldn't probably be able to save all of the data that LinusP does, but you can definitely inspect the current state of the market, the traded range for a horse in the past X seconds and trigger things accordingly (bets, save to db, Telegram alerts, etc).

[johnsheppard]

Thanks LinusP. looks cool! I think your tools highlights a big difference between c# world (microsoft) and python world.

In a Microsoft world, you pretty much live in visual studio and most of the technology is Microsoft centric... it has its pro's and cons... in python looks like a bunch of different companies

I haven't been doing this as long as LinusP (and i'm not profitable yet either), but here are some of the things that I use;
- WPF,/Xaml, Devexpress UI controls and their MVVM framework, MVVM patterns
- Entity Framework (code first, no stored procs)
- MS-SQL Server....(thinking of ditching this for just plain XML files)...
- C#
- Selenium for webscraping
- MS Test Framework for unit testing

I like that I more or less can just do it all in visual studio. It keeps my anxiety at bay...but that's just horses for courses...

A lot of this stuff is opensource, and I think Microsoft do realise that it has them beat...myself I'm all for paying for whatever is best value open source or not.....I would suspect your commercial options are a whole lot better in Microsoft land...(that's just half assed speculation...dunno really)

For those that haven't been there and done this....each of these technologies there are usually 4 or 5 leading options on the market that need to be analysed and decided on...it takes ages to do all that stuff...(e.g. MVVM frameworks might be Caliburn Micro, Prism, MVVMLight, and a gazillion others, ORMS might be dapper, entity framework LLBLGEN, or a gazillion others, UI can be Winforms, WPF...or HTML if you wanna go webbased)

There are also plenty of opensource libraries around which might be useful: Tulip Indicators (for financial indicators)...deedle for time series analysis, etc... (I don't use these)

IMO, The best thing you can do is keep it simple...don't have a UI, don't have a database, don't have anything until you really need it...work on your domain models first and foremost...(one of my many mistakes)

And if you are deciding on a language or platform...I don't know for sure...but I reckon...it doesn't really matter....6-7 years of hard gaff is what is more important

Or maybe just use Betangel

Hope this information is useful to someone...

[spreadbetting]

In regards to GUI’s this is not what python is for, tkinter will make you want to kill yourself. Instead you should look at building a web app on top of an API or database, this is my setup:

That's a pity, when I started botting (pre api) I choose php simply because I needed some application that was free and able to scrape webpages. Php was the obvious choice as it was free and hundreds of coding smippets around as it was used to drive a lot of websites. The fact I could echo or print data to screen was a real bonus to me when the API came long to understand the structure of the returned data. Once things are running smoothly I happily lave them to it but some still echo to screen and run off meta refreshes so you can see how old they are.

Hardest part I've always found with botting is actually connecting with the API and the placebet/cancel/ bet routines, once you have those in place the rest is usually easily sorted with a few web searches. So the fact you've done the hard work and stuck the engine up for free is a real temptation to finally upgrade my coding. I'll have to have a look at tkinter or see if I can find some way of connecting python to a web page for debugging etc

[Euler]

I was going to learn Python to help my kids and encourage them into coding. What is the best IDE for it, in the informed wisdom of the forum?

As in, I want something that they could get on with, having never coded before.

[northbound]

I was going to learn Python to help my kids and encourage them into coding. What is the best IDE for it, in the informed wisdom of the forum?

My take: you don't need an IDE, especially at the beginning. All you need is a text editor with syntax highlighting.

My editor of choice is Sublime Text which is not free. A free one I could recommend is Atom.

[LinusP]

I was going to learn Python to help my kids and encourage them into coding. What is the best IDE for it, in the informed wisdom of the forum?

As in, I want something that they could get on with, having never coded before.

PyCharm, combined with an anaconda distribution it will give you all the libraries you need whilst you learn. Great debugger and interactive console as well.


Yes I am on the extreme end when it comes to infrastructure costs but I have worked with teams where 5 figures per month are normal. But then I know one guy who spends £25 a month and is very profitable! For my first year I created an AWS account and never went more than £5 above the free tier for a year.

[ShaunWhite]

Wow you guys have some sophisticated setups. In comparison mine seems quite straightforward. Visual studio/C#, SQL... That's it.

Like LinusP I store the entire api stream but I move the data to my local machine daily (2gb?) so I don't need much server capacity. 2 servers costs me about 80 quid a month, each running multiple instances of my exe with parameters passed in via config files to control the various tasks it can perform, Collection only/Sim mode(from live or stored api data) /Live mode/strategy type/strategy params etc. All scheduled using TaskScheduler.

No UI, and I monitor progress & status etc using text logs which get pulled into Excel and viewed via RDP. If I did any scrapping or had a web app for monitoring it might be be more complex, but I don't, so it isn't.

[ShaunWhite]

IDE? Visual Studio community edition. (free)
Everything in one place and a vast amount of available info/samples etc. KISS.

[George Bets]

Quite an interesting topic this. I regularly consider using the API as well, but never actually took this step.

All my strategies are run in Python, and this Python program communicates with the Excel sheets from BetAngel. Although this works for a limited number of markets at the same time, it is not an ideal situation. Especially if I want my trading system to be fast.

This topic makes me think about the API again. I know the it will eventually be the best decision, but it takes time to develop everything from scratch.

[PeterLe]

I was going to learn Python to help my kids and encourage them into coding. What is the best IDE for it, in the informed wisdom of the forum?

As in, I want something that they could get on with, having never coded before.

Peter
Check out the udemy course on the first post of the thread; very good and cheap (£13)
Ive started to sublime text on their recommendation; not sure how long Ill be able to use it without registering though..

[PeterLe]

I was lucky that I started learning when the new API was being rolled out..................

Bloody Hell Liam that is impressive
No wonder my profits have gone down over the years
Inspirational though as to what can be achieved and thanks for sharing the code
Regards
Peter

[johnsheppard]

No UI, and I monitor progress & status etc using text logs which get pulled into Excel and viewed via RDP. If I did any scrapping or had a web app for monitoring it might be be more complex, but I don't, so it isn't.

IMO, this the way to do it...at least in Microsoft world...simple is best...otherwise you spend your life engineering instead of making money...

My own app is partly the way it is because I came to trading last...I started out thinking I could make an edge from staking plans and abusing online casinos, then started automating matched betting, and a little bit of arbing before I realised there's no way I can scale that stuff......that's why I have a UI...and it's useful, but has overheads...

But I presume this is the path many go through when they are raised by parents to be risk averse non gamblers...

[mcgoo]

In my option in python is the best language to learn as a beginner, there are so many brilliant open source libraries available, requests / pandas / scipy / Django. It allows you to do a lot but with minimal experience, this of course comes at a cost due to it being a high level language it is never going to be as quick as C.

I was lucky that I started learning when the new API was being rolled out, I had 2/3 strategies running on BA that I migrated across to python in a month (from zero programming experience) I then soon realized that the open source wrappers available were a load of crap and not maintained so wrote my own. I have then just kept on top of changes and was one of the first to develop streaming which opened up a lot of doors!



This is my web app which I use to monitor things whilst lying on the beach

55740446-C6B3-4AF1-9A2E-D008C8A869BD.png

Damn that's impressive.Congrats. You are what I want to be when I grow up ..maybe in the next life
Seriously well done...inspirational stuff