The Devil is in the Definitions

In the aftermath of the media frenzy around Australia’s most successful soccer team, The Matildas, making it to the finals of the World Cup, the Australian Football League made a grab for the headlines with the exciting announcement that women’s football would finally get the same prize money as the men.

It must be true. It’s all over the media. Check it out.

A screenshot of 5 headlines from 5 different media sites, all saying variations on the theme: "AfL announces equal prize money for competitions"

But wait! There’s more!

A different screenshot of 5 different headlines from 5 new media sites, all saying variations on the theme: "AfL announces equal prize money for competitions"

When you actually look into the detail, though, it turns out that the word “equal” is doing some unreasonably heavy lifting here. The pool of prize money for the AFLW is, in fact, equal to the pool of prize money for the AFL (which is still not referred to as the AFLM, for some bizarre reason). Unfortunately that money is shared between the top 4 men’s teams, while in the AFLW it is shared between the top 8 teams.

This of course means that winning teams receive half the amount in the AFLW compared to winning teams in the AFLM (yes, I will call them that, sue me). You don’t exactly need a Masters in Data Science to figure that out. You also don’t need a degree in critical thinking to know that this is not equality.

Now, I am 51 and have only been to one football game in my whole life, so I am in no position to discuss the reasons for the decision. Maybe there is some logic behind it. Certainly the women’s teams have been operating on a relative shoestring for years, so spreading the money out over more teams might be reasonable. Or, just speculating here, perhaps we could do more than “equalise” the prize money. Perhaps we could equalise the pay and the operating budgets at the same time. Perhaps, heresy I know, we could even pay the women’s teams MORE for a while to help them build up to the same level of resources as the men’s. (I’m going to refrain from going into a rant about whether sportsmen deserve those levels of pay at all.)

Things could be worse. Last year top tier AFLW players received a 94% pay rise that put them close to $72,000 per year. Some top tier AFLM players, for reference, received over a million dollars in the same year. Only ten men across the whole league received less than $100,000. The average salary in the men’s league was $406,000.

Now look, I’m not here to rant about inequality in sport, though inequality of any kind does tend to make steam come out my ears. The point is that, if you create your definitions carefully, you can make all kinds of accurate but wildly misleading statements and have the world mostly swallow them whole.

This is a big part of the reason why I run a Data Science Education Charity, and while all of our resources and projects use real, complex, flawed datasets. Because we’re working towards a world where everyone looks beyond the headlines and asks what the definitions are.

  • What do you mean by “Equal”? As we’ve seen, the women are being reported as receiving equal prize money while they receive half as much per team as the men. I’m no expert, but I’m fairly sure the teams do not contain half as many players!
  • What data are you including in your definition, and what are you excluding? Are you including salaries, or just prize money? If we’re talking salaries, are you including players at all levels? Are you including just top ranked, or low ranked, or excluding the highest salaries?
  • If you are comparing two groups, are you using equivalent measures for both groups? Total prize money might look very different if it includes a different number of games, or, indeed, is spread over a different number of teams or players.
  • What happens if we expand the definition? What if we include coaches and other staff? Choosing which salaries to include can radically change the story you tell.
  • What year is this data for? How has it changed? Are you using last year’s data because this year’s is particularly egregious?
  • If you’re comparing, what conditions differed between the two years? Have you chosen to compare to a year before the salary cap came in, so that the rise doesn’t look so bad? Was one of the years full of lockdowns that meant no in-person crowds?

I recorded an episode of Make Me Data Literate on Monday with the amazing Larene Le Gassick, who recommended the book “How to Lie with Statistics”. That book is an excellent education in this sort of chicanery. I don’t expect everyone to have read it! But imagine if we taught all kids to think that way. Imagine if we all left school equipped to spot when data is being used to pull the wool over our eyes?

That’s the world adsei.org aims to build. Meanwhile, asking those tricky, sceptical questions is something we can all get better at, with practice.

Leave a Reply