Football Data (CSV, JSON) - UPDATED 16/08/17

Post Reply
dm1900
Posts: 71
Joined: Sun Jan 15, 2017 10:02 pm

Euler wrote:
Thu Aug 31, 2017 12:21 pm
Corners hold a relationship with goals. So if you plot goals you also plot corners.
Hmm but surely corners can add more information to game state - there may be two games that are 1-0, except the first game has 10 corners by the home team whereas the second only 2 by the home team. If we think about expected goals in the second half, doesn't the 10 corners indicate aggressive play and so more of an expectation of goals in the first game rather than the second?
User avatar
Euler
Posts: 24700
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

Describe how a corner would occur.
dm1900
Posts: 71
Joined: Sun Jan 15, 2017 10:02 pm

Corner = A ball going over the goal line but last touched by the defending team. So usually via clearance or a save by the goalie - both of which indicate that the attacking team are in dangerous positions. So more corners = more attempts to attack by a team, whereas the number of actual goals may not be so indicative as shown by my example.
User avatar
Euler
Posts: 24700
Joined: Wed Nov 10, 2010 1:39 pm
Location: Bet Angel HQ

So if you look at a corner it's more or less linked to an attacking move and can be related to shots on and off target. So it's effectively a derivate of shots (in general) which is a derivative of goals.
footysystems
Posts: 165
Joined: Thu May 09, 2013 5:57 pm

Corners can be misleading imo the best way to predict goals or expect goals is to divide the number of dangerous attacks by the number of attacks to give you a % > than 70% high expectancy of a goal.
JamieS
Posts: 2
Joined: Sat Jun 03, 2017 6:10 pm

A massive thank you Welshboy for taking the time to do this and share it.
dm1900
Posts: 71
Joined: Sun Jan 15, 2017 10:02 pm

It is strange since there are no goals past 90' - I imagine everything in extra time is just given a 90' time anyway?
User avatar
Dallas
Posts: 22672
Joined: Sun Aug 09, 2015 10:57 pm
Location: Working From Home

doovd wrote:
Thu Sep 07, 2017 6:50 pm
It is strange since there are no goals past 90' - I imagine everything in extra time is just given a 90' time anyway?
All goals that occur in added time will get counted as either 45 or 90
Tenable
Posts: 20
Joined: Sat Jul 16, 2016 4:04 pm

spreadbetting wrote:
Wed Aug 23, 2017 10:18 pm
I stuck welshboy's data up on a simple website, I'll try and make it a big more searchable when I figure out php7 :(
Enjoying having a play about with this SB,, just wondering if your still going too add more search options too it at some stage?
elliotarcher1
Posts: 3
Joined: Thu Nov 09, 2017 4:34 pm

Hi Guys,

I'm doing a bit of analysis on stats at halftime/fulltime but I'm falling short trying to find halftime data without scraping it manually. Does anyone have halftime shots/shotsOT/Odds data to hand? Or perhaps know where I might be able to get hold of this? I'm only really interested in the top leagues. Would be seriously useful :D

P.s Loving this thread, loads of interesting chat going on
User avatar
Kafkaesque
Posts: 886
Joined: Fri Oct 06, 2017 10:20 am

The best part of being a relative newbie on here is these brilliant topics from before my time which pop up every so often :D

Brilliant work, Welshboy!

An ever so slightly off topic question for ya. Whoscored.com has some stats, I would love to delve more into. Before I try to teach myself to scrape data, though, am I correct in assuming that they've specifically coded the site to make it highly problematic to scrape it for data?
elliotarcher1
Posts: 3
Joined: Thu Nov 09, 2017 4:34 pm

Whoscored happens to be the website I value the most but yes, unfortunately, it is very difficult to scrape!

Betfair do offer what I am trying to find (historical HT odds) but I am not really sure I am willing to pay £200/month for it at this stage in my trading career!

Ah well...
Angelbaby
Posts: 55
Joined: Wed Nov 08, 2017 7:49 pm

Thanks to the poster on this one good stuff for us newbies.

I have a question if I am looking to back the draw in a certain match and want to work out roughly when the first 1st goal would be scored.

Would you seasoned football traders look at the specific games involving the 2 teams (as this could be only 2 matches) or would you look at all games played by each team in this league and then take an average from that wider sample?
rolandimone
Posts: 4
Joined: Mon Nov 06, 2017 4:47 pm

welshboy06 wrote:
Sun Aug 13, 2017 9:45 am
Hi All,

UPDATED 16/08/17
ADDED CSV VERSIONS OF ALL LEAGUES

I've decided to start collecting football data. Mainly because I have an interest in the sport and also because of a certain jonnyg throwing stats around in a very hard to read format.

The data is quite simple and was just scraped from HKJC.

The data is in json format, but you should be able to convert it to csv and import it to Excel. I chose json as it reduces data duplication, is structured and can be read reasonably well by the human eye. It also works really well with Python (Which I'm using to scrape and also analyse the data)
The data contains the following info.

Code: Select all

League
Season
Game Date
Home Team
Away Team
And also Goals and Red cards (Player, Time and Team)
-----
Soon:
Total FT/HT Goals and Red Cards for HT and AT (Included in CSV's)
I've managed to scrape ALL leagues and Seasons from HKJC (Where specific goal time data is provided)
I may decide to scrape more data, but for most leagues there is 10+ years worth of games, with goal and red card time data.

The data is a little too big to post here, and I didn't fancy splitting everything into multiple zip files. So I've uploaded and shared to my Dropbox
Not sure if the mods/admins could increase the file size limit, so I can add directly to the thread.
The below is a direct link to the folder containing all the data I've scraped.
Each league is in a separate text file, and I've also included a .zip of ALL the files.

https://www.dropbox.com/sh/7sif9n90ehys ... bd6ba?dl=0

Contents of the above files

Code: Select all

Number of Countries: 20
Country: Argentine - Leagues: 1
League Name: Argentine Division 1 - Seasons: 11
(2009-10(Au), 2010-11 (Sp), 2010-11(Au), 2011-12(Au), 2012-13 (In), 2012-13(Fi), 2013-14 (In), 2013-14(Fi), 2014-15 (In), 2015, 2016)
Country: Australian - Leagues: 2
League Name: Australian Division 1 Playoffs - Seasons: 7
(2008-09, 2009-10, 2010-11, 2011-12, 2012-13, 2013-14, 2014-15)
League Name: Australian Division 1 - Seasons: 9
(2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Belgian - Leagues: 3
League Name: Belgian Division 1 Championship Playoffs - Seasons: 1
(2016)
League Name: Belgian Division 1 UE Cup Playoffs - Seasons: 1
(2016)
League Name: Belgian Division 1 - Seasons: 2
(2015-2016, 2016-2017)
Country: Brazilian - Leagues: 3
League Name: Brazilian Division 1 - Seasons: 9
(2005, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016)
League Name: Brazilian Paulista League Knockout stage - Seasons: 1
(2014)
League Name: Brazilian Paulista League - Seasons: 1
(2014)
Country: Chilean - Leagues: 1
League Name: Chilean Division 1 - Seasons: 6
(2014-15(AP), 2014-15(CL), 2015-16(AP), 2015-16(CL), 2016-17(AP), 2016-17(CL))
Country: Dutch - Leagues: 3
League Name: Dutch Cup - Seasons: 7
(2005-06, 2007-08, 2008-09, 2009-10, 2010-11, 2011-12, 2012-13)
League Name: Dutch Division 1 - Seasons: 10
(2005-2006, 2006-2007, 2008-2009, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
League Name: Dutch Division 2 - Seasons: 6
(2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Eng - Leagues: 4
League Name: Eng Championship - Seasons: 13
(2004-2005, 2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
League Name: Eng League 1 - Seasons: 8
(2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
League Name: Eng League Cup - Seasons: 6
(2006-07, 2007-08, 2009-10, 2010-11, 2012-13, 2013-14)
League Name: Eng Premier - Seasons: 14
(2003-2004, 2004-2005, 2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: French - Leagues: 4
League Name: French Division 1 - Seasons: 12
(2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
League Name: French Division 2 - Seasons: 6
(2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
League Name: French FA Cup - Seasons: 1
(2005-06)
League Name: French League Cup - Seasons: 1
(2008-09)
Country: German - Leagues: 3
League Name: German Cup - Seasons: 5
(2006-07, 2007-08, 2008-09, 2012-13, 2013-14)
League Name: German Division 1 - Seasons: 14
(2003-2004, 2004-2005, 2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
League Name: German Division 2 - Seasons: 8
(2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Italian - Leagues: 2
League Name: Italian Cup - Seasons: 2
(2005-06, 2007-08)
League Name: Italian Division 1 - Seasons: 14
(2003-2004, 2004-2005, 2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Japanese - Leagues: 3
League Name: Japanese Division 1 - Seasons: 14
(2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015-Stage 1, 2015-Stage 2, 2016-Stage 1, 2016-Stage 2)
League Name: Japanese Division 2 - Seasons: 9
(2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016)
League Name: Japanese League Cup - Seasons: 9
(2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014)
Country: Korean - Leagues: 1
League Name: Korean Division 1 - Seasons: 1
(2016)
Country: Mexican - Leagues: 1
League Name: Mexican Premier - Seasons: 6
(2014-15 (AP), 2014-15(CL), 2015-16(AP), 2015-16(CL), 2016-17(AP), 2016-17(CL))
Country: Norwegian - Leagues: 1
League Name: Norwegian Division 1 - Seasons: 12
(2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016)
Country: Portuguese - Leagues: 1
League Name: Portuguese Premier - Seasons: 9
(2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Russian - Leagues: 1
League Name: Russian Premier - Seasons: 3
(2014-2015, 2015-2016, 2016-2017)
Country: Scottish - Leagues: 2
League Name: Scottish League Cup - Seasons: 5
(2005-06, 2007-08, 2008-09, 2009-10, 2012-13)
League Name: Scottish Premier - Seasons: 12
(2005-2006, 2006-2007, 2007-2008, 2008-2009, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Spanish - Leagues: 1
League Name: Spanish Division 1 - Seasons: 13
(2003-2004, 2004-2005, 2005-2006, 2006-2007, 2007-2008, 2009-2010, 2010-2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017)
Country: Swedish - Leagues: 1
League Name: Swedish Division 1 - Seasons: 12
(2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016)
Country: US - Leagues: 2
League Name: US Football League Playoffs - Seasons: 8
(2007, 2009, 2010, 2011, 2012, 2013, 2014, 2015)
League Name: US Football League - Seasons: 10
(2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016)
Okay so there are now two sets of CSV files.
One with reds and goals separated, and one where they are together as "Events" The columns for each are as follows:


Goals and Reds Separated

Code: Select all

SEASON, DATE, HOME, AWAY, HHT_GOALS, AHT_GOALS, HFT_GOALS, AFT_GOALS, TOTAL_GOALS, HOME_REDS, AWAY_REDS, TOTAL_REDS, RED_1_TIME, RED_1_TEAM, RED_2_TIME, RED_2_TEAM, RED_3_TIME, RED_3_TEAM, RED_4_TIME, RED_4_TEAM, RED_5_TIME, RED_5_TEAM, RED_6_TIME, RED_6_TEAM, RED_7_TIME, RED_7_TEAM, RED_8_TIME, RED_8_TEAM, RED_9_TIME, RED_9_TEAM, RED_10_TIME, RED_10_TEAM, RED_11_TIME, RED_11_TEAM, RED_12_TIME, RED_12_TEAM, RED_13_TIME, RED_13_TEAM, RED_14_TIME, RED_14_TEAM, RED_15_TIME, RED_15_TEAM, GOAL_1_TIME, GOAL_1_TEAM, GOAL_2_TIME, GOAL_2_TEAM, GOAL_3_TIME, GOAL_3_TEAM, GOAL_4_TIME, GOAL_4_TEAM, GOAL_5_TIME, GOAL_5_TEAM, GOAL_6_TIME, GOAL_6_TEAM, GOAL_7_TIME, GOAL_7_TEAM, GOAL_8_TIME, GOAL_8_TEAM, GOAL_9_TIME, GOAL_9_TEAM, GOAL_10_TIME, GOAL_10_TEAM, GOAL_11_TIME, GOAL_11_TEAM, GOAL_12_TIME, GOAL_12_TEAM, GOAL_13_TIME, GOAL_13_TEAM, GOAL_14_TIME, GOAL_14_TEAM, GOAL_15_TIME, GOAL_15_TEAM

Goals and Reds Together as Events

Code: Select all

SEASON, DATE, HOME, AWAY, HHT_GOALS, AHT_GOALS, HFT_GOALS, AFT_GOALS, TOTAL_GOALS, HOME_REDS, AWAY_REDS, TOTAL_REDS, EVENT_1_TIME, EVENT_1_TEAM, EVENT_1_TYPE, EVENT_2_TIME, EVENT_2_TEAM, EVENT_2_TYPE, EVENT_3_TIME, EVENT_3_TEAM, EVENT_3_TYPE, EVENT_4_TIME, EVENT_4_TEAM, EVENT_4_TYPE, EVENT_5_TIME, EVENT_5_TEAM, EVENT_5_TYPE, EVENT_6_TIME, EVENT_6_TEAM, EVENT_6_TYPE, EVENT_7_TIME, EVENT_7_TEAM, EVENT_7_TYPE, EVENT_8_TIME, EVENT_8_TEAM, EVENT_8_TYPE, EVENT_9_TIME, EVENT_9_TEAM, EVENT_9_TYPE, EVENT_10_TIME, EVENT_10_TEAM, EVENT_10_TYPE, EVENT_11_TIME, EVENT_11_TEAM, EVENT_11_TYPE, EVENT_12_TIME, EVENT_12_TEAM, EVENT_12_TYPE, EVENT_13_TIME, EVENT_13_TEAM, EVENT_13_TYPE, EVENT_14_TIME, EVENT_14_TEAM, EVENT_14_TYPE, EVENT_15_TIME, EVENT_15_TEAM, EVENT_15_TYPE, EVENT_16_TIME, EVENT_16_TEAM, EVENT_16_TYPE, EVENT_17_TIME, EVENT_17_TEAM, EVENT_17_TYPE, EVENT_18_TIME, EVENT_18_TEAM, EVENT_18_TYPE, EVENT_19_TIME, EVENT_19_TEAM, EVENT_19_TYPE, EVENT_20_TIME, EVENT_20_TEAM, EVENT_20_TYPE, EVENT_21_TIME, EVENT_21_TEAM, EVENT_21_TYPE, EVENT_22_TIME, EVENT_22_TEAM, EVENT_22_TYPE, EVENT_23_TIME, EVENT_23_TEAM, EVENT_23_TYPE, EVENT_24_TIME, EVENT_24_TEAM, EVENT_24_TYPE, EVENT_25_TIME, EVENT_25_TEAM, EVENT_25_TYPE, EVENT_26_TIME, EVENT_26_TEAM, EVENT_26_TYPE, EVENT_27_TIME, EVENT_27_TEAM, EVENT_27_TYPE, EVENT_28_TIME, EVENT_28_TEAM, EVENT_28_TYPE, EVENT_29_TIME, EVENT_29_TEAM, EVENT_29_TYPE, EVENT_30_TIME, EVENT_30_TEAM, EVENT_30_TYPE
Please let me know if I should make any changes to the below columns, or the CSV's at all
I've tried, best I can, to go through the data and make sure all games are there. I've not spotted any leagues with missing games as of yet. BUT PLEASE LET ME KNOW IF YOU SPOT ANY ISSUES.

Just to note: The data is all scraped from HKJC, so if there are any errors it would be down to the data they provided. (The dates I've been told are in HK time)

Cheers,
Adam
Link not working.
User avatar
Dallas
Posts: 22672
Joined: Sun Aug 09, 2015 10:57 pm
Location: Working From Home

rolandimone wrote:
Thu Nov 23, 2017 12:02 pm

Link not working.
I've just tried it and its still active and working
Post Reply

Return to “Betfair Data”