These ELO rankings have evolved over the past few years. There was a bit (ok, a lot) of a sloppy start, porting over the math from the Brotherly Game articles, complete with starting points & game weights, and then trying to push the official Canadian division rankings into position. That first port-over required modifications to L1O, PLSQ, & PCSL on several occasions and it still wasn’t quite right at the end of 2016. There have been numerous Twitter, Voyageurs, Facebook, and comment conversations with some of the heavy-weights in the Canadian Soccer blogosphere that have helped push some of the early changes, but I never fully implemented their ideas completely because I always had hope that the new Canadian leagues would improve enough to meet the level where they were placed. More recent discussions with some of the team coaches, association presidents, and others in the field have helped point to some of the unique factors that need to be considered, and just how important (or unimportant) certain competitions to their clubs.
Did the Canadian 3rd Division leagues improve? Yes. They are substantially better today than they were in 2014. Are they now equivalent to last year’s USSF Div-3 USL? No. PDL? Sorry, not yet. Are they even equivalent to each other? Close enough. This means more tinkering with starting points.
There are other things that have thoroughly bothered me about the initial math. For one thing, is the home field advantage really huge enough to warrant a 100-point gap-change? Maybe if FC London were suddenly to play at BC Place, but probably not TFC at BC Place or even at TD Place in Ottawa. The quality of stadiums, even at the lower levels, has greatly improved, so the “home field advantage” probably shouldn’t be as large.
The game weighting was, at best, a logical subjective choice, but the math doesn’t take home & away aggregate competitions into account. In these competitions, that 2nd match is more important. We have to correct the ELO math formula to take this into account, and we wanted to make the weighting of matches more objective, so that if a new competition or league (CPL & BCRT3, we’re talking about you) were to be added, we can quickly fit in the appropriate requirements and it’ll spit out a more-or-less appropriate weight. To add to the complication, the PLSQ is a very special league since it doesn’t have playoffs, requiring some adjustments its weighting of matches.
There are a lot of little tweaks coming to the formulas, starting points, and game weights for the 2017 season. We’ll need to break it down. I’m using 3-tabs, one for the Brotherly Game original, one for the 2014-2016 methodology used here on NSXI, and a third to explain what we’ll be using for 2017 and moving forward.
Before I get into the changes, I’d like to go over what ELO-ratings are, as well as the advantages & disadvantages of ELO-ratings. This is basically a restating & simplification of the Brotherly Game article, so feel free to review that for additional details & skip down to “starting points”.
ELO-ratings Introduction:
In 1997, Bob Runyan adapted Hungarian-American mathematician Dr. Árpád Élő’s system for rating chess players to international football. Since then, the system has been adapted for use by footballdatabase.com to rank teams across the world, and FIFA has modified it for use in ranking Women’s International Soccer.
In turn, Brotherly Game used a modified version that in-turn accounted for the unique qualities of the beautiful game in North America to account for the unique league and competition structures in the US. At the time of NSXI’s adoption, Brotherly Game used data streaming from 2011-2013, including 4,600 competitive matches, for determining their weights and the current ratings of teams in Canada & the United States.
NSXI adopted Brotherly Game’s method, with a few minor tweaks in 2014, making modifications & additions to include Canada’s D3 leagues (PLSQ & L1O) and the PCSL. There were also some modifications to weights and the like to better adapt to the Canadian circumstances. The initial ratings caused a stir, and starting points have been adjusted for the Canadian-only leagues at several points.
The Basics
ELO was adapted to soccer by weighting for the type of match, adjusting for the home team advantage, and adjusting for goal difference in the match result.
Post-match, each team’s ELO-ratings is adjusted based on:
- The team’s old rating
- The importance or weight of the match
- The goal difference of the match
- The result of the match including home field advantage
- The expected result of the match
Strengths:
ELO is based primarily on objective measures, and uses a team’s historical prowess to help determine their relative rating. Every match’s results, including the goal difference, is used to calculate the new ratings. Those match results are weighted based upon both the importance of the match/competition and the expected outcome of the match based upon each team’s history in the competition.
Weaknesses:
The weight or importance of the matches are decided by the ratings-designer. Therefore there is a subjective element to the weights of the match. Explanation on how we’ve addressed this can be found below under “Game Weights.”
Each team must start with a score, which is chosen by the ratings-designer. Again, a subjective element. This has been minimized by Brotherly Game by using the US Open Cup results to develop starting points. We’ve used their starting point, but adjusted based upon the official division of the league, and then used the more recent US Open Cup, Canadian Championship, & Interprovincial Cup results to modify those starting points.
There is a goal difference factor. The original was very strong, so we’ve utilized the same as Brotherly Game. A team that wins a game by two goals will get 50% more points for that outcome than if they had won by 1 goal. A three goal difference gets 75% more than 1 goal difference.
The formula & variable definitions:
This is almost 100% as per Brotherly Game.
Rn = Ro + K*G (W – We)
Where;
Rn = The new team rating
Ro = The old team rating
K = Weight index regarding the tournament of the match
G = A number from the index of goal differences
W = The result of the match
We = The expected result
Goal Differential = G
The number of goals is taken into account by use of a goal difference index. G is increased by 25% if a game is won by two goals, and if the game is won by three or more goals by a number decided through the appropriate calculation shown below;
If the game is a draw or is won by one goal
G = 1
If the game is won by two goals
G = 1.25
If the game is won by three or more goals
G = (11+N)/10
Where N is the goal difference
Result of the Match = W
W is the result of the game (1 for a win, 0.5 for a draw, and 0 for a loss).
Expected Result of Match = We
We is the expected result (win expectancy with a draw counting as 0.5) from the following formula:
We = 1 / (10^(-dr/400) + 1)
dr is defined below, as it is being altered. Essentially it accounts for home field advantage while obtaining the difference in rating between two teams.
Starting Points:
Original from Brotherly Game
MLS – 1400
NASL – 1200
USL – 1100
PDL – 1000
Other – 1000
These guys also had MLS Reserve, but in 2014 the reserve teams were merged into USL. Their rankings also completely ignored PLSQ, L1O, PCSL, and when the data was downloaded, seemed to miss some Canada-only matches.
2014-2016 NSXI
MLS – 1400
NASL – 1200
USL – 1000
PDL – 900
PLSQ – 900
L1O – 900
PCSL – 800
Other (for US Open Cup) – 600
I gave more space between USL & NASL, and maintained the PDL/USL space. Originally, L10 & PLSQ were equal to USL, but that has been slowly correcting as I’ve had the chance to see more matches. Brotherly Game also had unranked teams at 1000, so I think my low-balled 600 is far more reasonable.
New Methodology
MLS – 1400
NASL/USL – 1225
PDL/NPSL – 1000
L1O/PLSQ – 900
Other – 600
With USL now considered Division 2, I’ve matched it to NASL. This jump made the gap between PDL/NPSL & USL far too large to leave it at 900. As a result, I pushed it higher to 1000. L1O/PLSQ were originally going to be downgraded, but given the boost to PDL/NPSL, they can remain at the 900 mark. PCSL has been removed completely. Since there are no Canadian Teams in the NPSL, its score is really only being used for the US Open Cup matches.
Game Weights:
Original from Brotherly Game
MLS Cup Final & Semis – 50
US Open Cup Final & Semis – 40
Canadian Champ Final – 40
MLS Playoffs – 40
Canadian Championship – 30
US Open Cup – 30
MLS – 30
NASL Playoffs – 30
USL Playoffs – 25
NASL – 21
US Open Cup Prelim – 20
USL – 15
PDL Playoffs – 15
PDL – 10
2014-2016 NSXI
MLS Cup Final & Semis – 50
MLS Playoffs – 45
Can Champ Final & Semi – 40
US Open Cup Final & Semi – 40
NASL Playoffs – 35
MLS – 35
US Open Cup – 30
CanChamp NASL Play-in – 30
USL Playoff – 25
NASL – 25
Interprovincial Cup – 18
L1O Cup – 15
PLSQ Cup – 15
L1O/PLSQ Playoffs (?) – 15
PDL Playoffs – 15
PCSL Playoffs – 15
USL – 15
L1O, PLSQ, PDL, PCSL – 5
I felt Brotherly Game was too US-centric and anti-NASL in their game weighting. I had to add in a fair bit, and slide some numbers toward the lower end in order to fit “everything” in. Was I happy with these weights? No, but I was happier with them than just using BG’s.
New Methodology
This is by far the biggest change for 2017, and I think it will be a great move. We will be using a mathematical formula for determining the weight of different types of matches. The new formula takes into account skill-level division, FA Cups, post-season, silverware awarding matches, OpenCup/Voyageurs Cup Qualification matches, Champions League qualification, and aggregate home & away matches. This also means that if the way a competition is played changes, say PLSQ adds a playoff format, a new league forms, or MLS ditches home & away aggregate games in its playoffs, then by changing a 1 or a 0 on the spreadsheet calculator, new weights are automatically generated.
K = 5*(5-(Div-2))+FA*5+Post*10+Silver*5+EQual*15+CQual*25+Agg1*(-5)+Agg2*(5)
Div = The highest skill-division playing. MLS =1, NASL/USL = 2, PDL = 4, L1O/PLSQ = 5, Unranked = 6. Because of the NASL/USL debacle, 3rd division is empty at the moment.
FA = Whether the match is a Cup match. Currently, only L1O & PLSQ have “FA Cups”. FA can either be a 1 or 0.
Post = Whether the match is a post-season match. Post can either be a 1 or 0.
Silver = Whether silverware is awarded as a result of the match. This means the MLS Final & Semi-finals, USL Finals & Semi-finals, PDL Final & Semi-final, L1O Final, L1O & PLSQ Cup Finals, Interprovincial Cup (home & away), US Open Cup Final, Voyageurs Cup Finals (home & away)… Basically, if you can put “final” at the end, it counts as a 1, otherwise, it’s a 0. There are two exceptions. The first is NASL, yes there is a final, but there is also a Spring & a Fall champion instead of conference champions. For PLSQ there is no post-season, and the final can be determined any game of the season. For both of these league, every game of the season has a 1 for Silver.
EQual = Entry qualification. If winning the match allows the team to take part in the Voyageurs Cup or US Open Cup, then this is a 1, otherwise it’s a 0. For our use, that means the Interprovincial Cup obtains the 1 since it is widely seen as the entry mechanism for L1O or PLSQ teams (even if not official as of yet).
CQual = Champions league qualification. If matches lead to the team to take part in Champions League, this is a 1, otherwise it’s a 0. This means for the Voyageurs & Open Cup, and the MLS Final, this will be a 1.
Agg1 & Agg2 = Home & Away aggregate game 1 & game 2. If the score accumulates via aggregate, then these are toggled 1 or 0 depending on the game. If results are not determined by aggregate, both of these will be 0. Since the 2nd match in an aggregate series is more important, that is now reflected in the weighting.
As a result of the new mathematical formula, the weights are as follows:
These are the game weights for 2017, barring any as-yet unannounced modifications to play/play-off structure. Will update image to reflect any changes.
9 Mar 2017 Edit:
Due to the reconfiguration of the CONCACAF Champions League, in 2017 there may be a 1-match play-in between the 2017 Canadian Champions & Toronto FC (2016 Canadian Champions). If this one-time match does occur, it will be weighted at 55, which is the average of the Voyageurs Cup 2-leg final.
Additionally, with the inclusion of the L1O & PLSQ Champions in the Voyageurs Cup, they have opted to discontinue the Interprovincial Cup. A sad development, but completely understandable given the higher-level of play being offered.
Home Field Advantage:
Original from Brotherly Game
100 points
dr formula:
(Home ELO+100)-Away ELO
&
Away ELO-(Home ELO+100)
Example (USL home vs MLS):
USL: (1100+100)-1400 = -200
MLS: 1400 – (1100+100) = 200
2014-2016 NSXI
100 points
I felt BG had a better grasp on this than I did, so left the gap as prescribed.
dr formula:
(Home ELO+100)-Away ELO
&
Away ELO-(Home ELO+100)
Example (USL home vs MLS):
USL: (1000+100)-1400 = -300
MLS: 1400 – (1000+100) = 300
New Methodology
25 points
+10 points per “division” jump
Most fields within the same league are near the same standard. If TFC plays at BC Place, they aren’t feeling a huge difference from their home. The crowd gives an edge, but the grounds aren’t hugely different.
However, if FC London goes to TD Place in Ottawa, there’s a huge jump. In this case, the HFA for Ottawa would be 25 (base) + 10 (L1O – PDL) + 10 (PDL – Div3) + 10 (Div3 – USL).
It also makes the largest gap between Unranked & MLS, 75.
A new formula for calculating dr is then required.
dr formula:
(Home ELO + (25 + (10 * |Home Div – Away Div|)))-Away ELO
&
Away ELO-(Home ELO + (25+(10* |Home Div – Away Div|)))
Example (USL home vs MLS):
USL:
(1025+(25+(10*|3-1|)))-1400
=(1025+45)-1400 = -330
MLS:
1400-(1025+(25+(10*|3-1|))) = 330
The “Big” Formulas:
Original from Brotherly Game
Rn = Ro + K*G (W – 1 / (10^(-dr/400) + 1))
dr formula:
(Home ELO+100)-Away ELO
&
Away ELO-(Home ELO+100)
2014-2016 NSXI
Rn = Ro + K*G (W – 1 / (10^(-dr/400) + 1))
dr formula:
(Home ELO+100)-Away ELO
&
Away ELO-(Home ELO+100)
New Methodology
Rn = Ro + K*G (W – 1 / (10^(-dr/400) + 1))
dr formula:
(Home ELO + (25 + (10 * |Home Div – Away Div|)))-Away ELO
&
Away ELO-(Home ELO + (25+(10* |Home Div – Away Div|)))
K-formula:
K = 5*(5-(Div-2))+FA*5+Post*10+Silver*5+EQual*15+CQual*25+Agg1*(-5)+Agg2*(5)