Is the NRL rugby league ladder biased?
The coronavirus has had large and unexpected effects on professional sports. Seasons have been cancelled, delayed or shortened; stars have played in empty stadiums; broadcasts have dubbed crowd noises for television audiences, while fans have been able to buy life-size cardboard cut-outs to fill the empty seats. (The fan response has been reasonably positive in some leagues, such as the German soccer Bundesliga and the US Major League Baseball. But the cardboard idea took a sour turn in the Australian national rugby league when a cut-out of mass murderer Harold Shipman made an appearance in the stands.)
I recently found out that the two main ‘footy’ codes in Australia, the AFL and NRL, have both shortened their seasons. For example, the rugby league season has been shortened to 20 rounds; in other words, each team will play every other team once, and will then play five teams a second time. It’s this last feature that sparked my interest as an economist. “Oo, a natural experiment,” I thought. “Some teams will get a fairer deal than others!”
Little did I know that some teams always get a fairer deal than others, in both the NRL and the AFL! I usually follow the European football leagues, where the draw is as fair as can be: each team plays every other team twice (once at home and once away). I had assumed things were the same in Australia. Instead, a quick search reveals that each of the 16 NRL teams plays 24 games a year (6 teams once and 9 teams twice), while each of the 18 AFL teams plays 22 games a year (12 teams once and 5 teams twice).
I’m not sure how I never noticed this before while I grew up in Australia. With a bit of nerdy excitement, I googled how the double-teams are chosen. Surely some sophisticated draft mechanism to even the playing field? It turns out that the formula has nothing to do with fairness and everything to do with money. Before each season, the NRL, its clubs and the broadcasters decide which fixtures will bring in the most fan revenue. So, for example, local derbies like Parramatta versus Canterbury or Sydney City versus South Sydney will feature twice, as will all-Queensland battles like Brisbane versus the Gold Coast.
As the ‘footy’ season enters the finals, the impact of COVID has been felt very differently between the two codes. For the 18 teams in the Australian Rules competition, the season has been reduced from the regular 22 matches to 17; in other words, they now play a single round-robin. Ironically, the COVID reductions have made the competition fairer by removing its bizarre “round robin plus five” schedule. For a change, the revised 2020 draw is an even playing field to determine the best teams.
For the NRL, the reduced 20-round season means that each team plays ten teams once and only five teams twice. Seeing as this change was made mid-season, you’d think this would have been a prime opportunity to make a fairer draw on the basis of current team performance. Instead, the decision seems to have been made for the same financial reasons. I’m as shocked as you.
When the draw came out, there were a few articles (mainly on betting sites) analysing the changes. There was a general feeling that the Sydney City Roosters were hard done by, while the Parramatta Eels got an excellent outcome. But this was based largely on pre-season team rankings, which, for example, picked the Brisbane Broncos as one of the premiership favourites. (Instead, the Broncos had their worst season in history.)
Still, I started to think about how one might ex-post adjust the NRL to make it a bit fairer, based on the current season alone. In other words, can we apply some sort of weighting in order to work out where the teams would have been ranked if there was a fairer system for the fixtures, such as a single or a double round robin?
There are a couple of complicated options depending on your definition of ‘fair’. But the simplest option is to treat each pair of games between teams that played twice as one mini-match. In other words, if team A beats team B the first time they play but team B beats team A in the return game, we count this as if they played a single draw. We can also re-weight the For/Against points as well so that the adjusted ladder looks as close as possible like a single round robin among the 16 clubs.
Here’s what the NRL ladder looked like after the 20th and final regular round ended:
| Position | Team | Points | FA_diff |
| 1 | Panthers | 37 | 299 |
| 2 | Storm | 32 | 258 |
| 3 | Eels | 30 | 104 |
| 4 | Roosters | 28 | 230 |
| 5 | Raiders | 28 | 128 |
| 6 | Rabbitohs | 24 | 169 |
| 7 | Knights | 23 | 47 |
| 8 | Sharks | 20 | 0 |
| 9 | Titans | 18 | -117 |
| 10 | Warriors | 16 | -115 |
| 11 | Tigers | 14 | -65 |
| 12 | Dragons | 14 | -74 |
| 13 | Eagles | 14 | -134 |
| 14 | Cowboys | 10 | -152 |
| 15 | Bulldogs | 6 | -222 |
| 16 | Broncos | 6 | -356 |
And here’s what the COVID-adjusted ladder looks like:
| Position | Team_adj | Points_adj | FA_diff_adj |
| 1 | Panthers | 28 | 216.5 |
| 2 | Storm | 23 | 212 |
| 3 | Roosters | 22 | 190.5 |
| 4 | Eels | 22 | 78.5 |
| 5 | Raiders | 20 | 77.5 |
| 6 | Rabbitohs | 19 | 146 |
| 7 | Knights | 17 | 33 |
| 8 | Titans | 14 | -83 |
| 9 | Sharks | 13 | -22 |
| 10 | Warriors | 13 | -77 |
| 11 | Tigers | 12 | -23 |
| 12 | Dragons | 11 | -47.5 |
| 13 | Eagles | 11 | -124.5 |
| 14 | Cowboys | 7 | -132.5 |
| 15 | Bulldogs | 4 | -188.5 |
| 16 | Broncos | 4 | -256 |
There aren’t many changes. The Roosters finish above the Eels, about which, given the first week of the finals, I doubt many people would complain. The only significant feature is that the Titans would have made the finals at the expense of the Sharks. Noteworthily, the Broncos, who have been absolutely awful, retain their rightful wooden spoon.
Even though the ladder looks very similar, I started to wonder whether this is always the case. For starters, there’s a smaller adjustment this season compared to other seasons due to the already-reduced length. Also, the ladder is quite spread out (meaning a higher standard deviation), so it’s less likely that adjustments actually matter.
So, I ran my analysis again. Here are the official ladders and the adjusted ladders for every season since 2013. Green means the club would have done better if the season was adjusted; red means the club would have done worse.
I was most interested to see if some team is consistently advantaged or disadvantaged by the system. Uninterestingly (but reassuringly for the NRL, I guess), most clubs ‘win some, lose some’. On the other hand, there are some teams that have definitely been better off or worse off thanks to the system. Interestingly, this year’s minor premiers, the Penrith Panthers, have been the main beneficiaries. They’re not affected this season (they still win the minor premiership by a long way). But out of the seven seasons before this one, they finish lower on the adjusted ladder a whopping five times, and remain unchanged twice.
(I’ve done these last calculations by hand, so there’s every chance I’ve made a mistake – let me know!)
Other net ‘winners’ of the current system include the Broncos, who would have been lower in four ‘adjusted’ seasons and better off in only one (4:1), the Roosters (4:1), and the Eels (3:1, but the one season that the Eels would have been better off from an adjustment was 2016, when they were infamously stripped of their competition points due to a salary cap breach, so you could argue that it’s really 3:0).
Which are the clubs that have been disadvantaged by the system? The biggest loser is the poor West Tigers team, who would have finished higher in four seasons (0:4). The Titans (1:4), Bulldogs (2:4), Raiders (1:3) and Rabbitohs (1:3) also have reason to complain. Strangely, it seems like the traditionally richest clubs in the competition, like the Broncos and the Roosters, have been the ones who have benefited the most, while the poorest ones like the Tigers and Titans have been up against it. Probably a coincidence.
| Team | Would have been better off | Would have been worse off |
| Broncos | 1 | 4 |
| Bulldogs | 4 | 2 |
| Cowboys | 1 | 2 |
| Dragons | 2 | 1 |
| Eagles | 2 | 0 |
| Eels | 1 | 3 |
| Knights | 2 | 3 |
| Panthers | 0 | 5 |
| Rabbitohs | 3 | 1 |
| Raiders | 3 | 1 |
| Roosters | 1 | 4 |
| Sharks | 3 | 4 |
| Storm | 1 | 2 |
| Tigers | 4 | 0 |
| Titans | 4 | 1 |
| Warriors | 2 | 3 |
Of course, some case matter more than others (such as the Titans missing out on the finals this year). The Tigers would have made the finals in 2019 (at the expense of the Broncos), the Raiders would have made the finals in 2017 (ousting the Panthers), and in 2014, the Warriors would have made the finals (bye bye Broncos) and the Manly Sea Eagles would have scored a famous Minor Premiership. The Titans should have made the finals in 2013 at the expense of their Queensland counterparts, the Cowboys.
What does this all mean? Not much. But it was a fun way to spend an afternoon.