Evidence Based Medicine

This is a sneak peek into Chapter 2 of Raising Heretics: Teaching Kids to Change the World, due out on August 1st, and available in all the usual places from that date.

___

When I was a kid, doctors were treated as demigods. Patients did what they were told, and trusted that medical treatment was always based on science and evidence. Despite a range of negative experiences with the medical profession over the last few years, my default response to health professionals is still one of trust, so I am always horrified when I look into the evidence base for particular treatments, or specific drugs, and discover the alarming lack of scientific rigour that underpins a lot of common medical treatments.

It is disturbing in itself that the term “Evidence Based Medicine” was first coined as late as 1991, by an academic by the name of Gordon Guyatt at McMaster University. It was not, initially, a way of practicing medicine. Instead, it was the name of a course designed to encourage medical students to make their practice more scientific.

If evidence based medicine was only just being talked about in the nineties, you have to wonder how medicine was practiced before that. Sadly, a startling amount of medical practice has historically been based on assumptions, untested theories, and arrogance. And much of it still is.

Consider the treatment of hip pain. In 2018, my daughter, Zoe, was diagnosed with acetabular retroversion and dysplasia, meaning her hip sockets were both too shallow, and facing the wrong way. She was sent to a physiotherapist for rehabilitation, to see whether her hip function could be fixed with the right strengthening exercises. We were exceptionally lucky that the physio she was sent to was Josh Heerey, who, at the time, was working on a PhD in hip problems. Being used to physios poking, manipulating, and making grand pronouncements on the basis of “feel”, I was fascinated in that first appointment to see Josh using a dynamometer.

A dynamometer is a device for measuring force, and Josh used it to measure Zoe’s strength in all directions. This meant that not only did Josh know for sure which muscles were weak and needed work, he was able to use the dynamometer on subsequent visits to measure Zoe’s progress. Unfortunately Zoe’s retroversion was severe enough that she needed surgery, but after 6 months of physio work, she was very strong, which made her recovery much easier. It also meant that her post-surgery rehabilitation was both scientific and effective, as Josh continued to measure her strength and prescribe exercises that directly targeted areas of weakness. After major hip surgery, Zoe is now running and jumping, with no sign of ever having had an issue, except for some trophy scars.

Meanwhile I started having hip pain, and went to a local physio. (Josh was quite some distance away, so I thought it would be quicker to see someone close by.) The local physio diagnosed bursitis, used a tens machine, ultrasound, heat treatment and massage, and after weeks of sessions I got precisely no improvement. In fact, if anything, I was getting worse. I asked Zoe’s surgeon whether her condition was hereditary, and he ruefully confirmed that yes, it was likely her malformed hips were a genetic gift from her mother.

The bad news was that I was too old for the surgery that had helped Zoe. Before too long I was seeing Josh, and competing with Zoe to be the most obedient patient and do all of the exercises as prescribed. It was hard work, but within 6 months the surrounding muscles were strong, and I had no more hip pain. (Unfortunately I then had an insane 3 months of travel that trashed the other hip, so the dynamometer and I are currently close friends again.)

Traditionally the need for various radical and invasive hip surgeries has been determined from damage seen in X-Rays and MRIs. This was not based on studies showing a relationship between scans and pain or functional impairment, or indeed on studies of the effectiveness of the surgeries. It was simply a “logical deduction”. A recent study of the relationship between hip pain and imaging results by Josh and his colleagues at LaTrobe University found that there was actually no correlation between pathology seen in imaging and actual impairment of the hip. Imaging of people with no pain and no impairment showed similar levels of damage to imaging of people with pain and impairment, and there was no correlation between imaging results and hip function. It turns out that a lot of hip pain, my own included, can be effectively managed (and indeed banished) using physiotherapy.

And that’s not an isolated finding. Various studies have found that different knee and back surgeries are no more effective than placebo – in other words, if patients think they have had repair surgeries, but in fact have only had the incision and a bunch of experiences to make them think they’ve had surgery, their recovery is just as good as those who have had the actual surgery. Nonetheless, these surgeries continue to be performed, and described to patients as successful cures. Josh’s evidence based approach to treating hip pain is not, unfortunately, the norm.

___

It’s encouraging to see, based on articles like this in the prestigious medical journal The Lancet, that we are starting to recognise and acknowledge some of the issues with the way we practice science in general, and medicine in particular. Hopefully this section of my book will become increasingly inaccurate. But the first step is to admit the problem. I’ve lost count of the doctors I have seen who have been defensive and arrogant when questioned. These days I use it as a kind of shibboleth. Any doctor who is not interested in discussing evidence and risk with me is not a doctor who is interested in achieving the best outcomes, and not someone I will bother seeing twice. There are more and more doctors who are happy to be questioned, and who welcome informed patients like me. But there are still far too many of the other kind, and we are a long way from a truly evidence based health system.

Understanding the data will help us understand the danger

In Australia right now, both Sydney and Melbourne are battling outbreaks of the delta variant of covid, and wondering why it just doesn’t seem to lie down and die the way the virus did in Australia last year. There are many reasons for our current problems – too, too many of them political – but a lot of it comes down to the R0 value, or how contagious this variant is.

Last year’s variant was not nearly as contagious – some estimates say delta is three times as contagious as that one – and so it was easier to put down. But what is R0, and why do we care?

R0 is known as the “basic reproduction number”, and simply represents how many people an infected person will infect. So an R0 of 2 means that each person with covid spreads it to 2 other people. That means the infection doubles at every step, and that’s bad. 1 person infects 2 others, they spread it to 4, who spread it to 8, etc. At the 6th step, we’re already at 64 new people being infected. Left unchecked, at the 10th step we’re at 1024 new people infected.

2
4
8
16
32
64
128
256
512
1024

Delta has an estimated R0 of 5. Every person with delta will likely spread it to 5 other people. That means 1 case becomes 5, those 5 infect 25 more, those 25 infect 125 more, and it escalates alarmingly fast. Left unchecked, at the 10th step we’re at nearly 2 MILLION new people infected.

1
5
25
125
625
3125
15625
78125
390625
1953125

If we graph the first 6 steps, we can see how an R0 of 2 compares with an R0 of 5.

New infections for R0 of 2 versus R0 of 5. R0 of 5 is a red line jumping sharply upwards, to over 3000 by step 6. The blue line for R0 of 2 barely registers on the graph, it is relatively low and flat.

Or, to put it another way, check out these little coronaviruses. Each step is 5 times the step before.

5 steps of transmission of the delta virus, with an R0 of 5. Each little covid image represents one new person infected

The good news is that we can effectively modify the R0. The delta variant’s R0 is only 5 if people are interacting in a way that allows the virus to jump from person to person. This is why contact tracing can be so effective – if we can identify all of the people who have had contact with an infected person BEFORE they become infectious themselves, then we can prevent the virus from spreading any further. If we can drop the R0 to under 1, meaning that, on average, people infected with delta are infecting less than one other person, then we can get it under control.

Contact tracing works if we can identify every single person an infected person has been near enough to to transfer the virus. The trouble is that delta makes that jump so quickly and easily that contact tracing needs to identify everyone an infected person has walked past in the street, especially if they were unmasked. This is obviously not achievable. Plus, even the most dedicated of us sometimes forget to check in, so that is not 100% reliable either.

Wearing masks helps, but only if they are properly fitted, worn properly, and washed or replaced regularly.

This is why we need strict lockdowns. Because lockdowns reduce the number of people we each come into contact with, thus effectively reducing the R0, and limiting delta’s opportunities to spread.

If we don’t get the R0 down, we can’t control the spread of the virus. It’s as simple as that.

And if you’re wondering why I stopped my little covids at step 5 in the picture above, it’s because step 6 is too big to fit on the screen.

Step 6 is 3125 new cases – too many to display onscreen (this is not the full set)

Once again, our safety relies on our leaders (and the public) understanding exponential growth. It’s not actually complicated, but few of us have ever needed to know how it works. Now, more than ever, we need to build our data literacy, so that we can understand the danger we are in.

For more reasons why we need to be data literate, and how to teach our kids to change the world, check out Raising Heretics, pre-order now, or buy online in all the usual places from August 1st.

Our lack of Data Literacy is literally killing us

The start of the outbreak of covid’s highly contagious delta variant in Sydney was greeted with the usual hubris by NSW policy makers. The magical gold standard contact tracers would fix it without any of those dreadful southern lockdowns. Once again, NSW would show the world that covid could be vanquished without cost.

The first six days of the outbreak were, indeed, not scary at all. Daily cases were low. Why worry? They went 2, 2, 2, 2, 5, 10. Small numbers. Easily handled by contact tracing. Nothing to fret about. Except, to anyone who knows what exponential growth is, the 2, 5, 10 progression is deeply worrying. That’s case numbers doubling every day. And though 10 is still a small number, doubling every day takes 10 to 20, 40, 80, 160 within just five days. (For a quick video explainer on exponential growth, check out this one that I made last year.)

The next day was 18, awfully close to 20. But then we had 11, and everyone breathed a sigh of relief. We must always remember that we know how many people have tested positive for covid. We don’t know for sure how many people have it. That’s why we need everyone to get tested as much as possible, to keep the “tested positive” as close to “people who have it” as possible. But disease spread is not as consistent or predictable as an exponential series in a textbook, so the one low number was not particularly useful. What matters is the trend. The next day, June 25th, 29 new cases were reported. Still not scary numbers, especially when we’ve been watching the rest of the world in the hundreds of thousands, and even Melbourne got up to nearly 700 per day. 29 was nothing to fret about.

But understanding that growth was making epidemiologists and maths nerds very nervous, especially because the delta variant is three times as infectious as the variant Melbourne was dealing with last year. Delta has an R0 of 5, meaning for every person infected, on average 5 more people will catch the disease. That’s exponential growth. 1 becomes 5, which becomes 25, then 125, etc. When Gladys Berejiklian expressed her relief one day that “only” 9 of the new cases detected the previous day were out in the community while infectious, those 9 cases were likely to translate to 45 new ones. That was definitely not good news.

On July 2nd, increasingly alarmed by what looked to me to be awfully like Melbourne’s second wave being played out up north, I graphed the Sydney outbreak against Melbourne’s numbers, and saw this.

Graph of Sydney's delta outbreak as at July 2nd against Melbourne's second wave. The two lines are very similar, and trending jaggedly upwards.
Sydney Outbreak as at July 2nd, graphed against Melbourne’s second wave

I kept producing those graphs, as numbers grew, and every time I tweeted them someone would say “but we have contact tracing” or “but we’re in lockdown earlier.” And that scared me, too, because Sydney’s “lockdown” was not lockdown as we knew it in Melbourne. Non-essential retail was not closed. Neither was childcare. And numbers were still rising.

I tweeted this graph on July 10th, with more Melbourne numbers, and noted that within days of numbers around 50, the numbers in Melbourne were over 100.

Graph of Sydney's delta outbreak as at July 10th against Melbourne's second wave. The two lines are very similar, and Melbourne goes sharply up only a few days later.

Later that day, 77 new cases were reported. The next day was 112. But still people were saying “it’s ok, we’re in lockdown.” So I made this graph, showing the types of restrictions introduced in Melbourne, against the case numbers.

Graph of Melbourne's second wave with restrictions marked in as they were applied. 10 postcodes entered stage 3 early in the curve. then two more postcodes and 9 public housing towers. Schooling went online for all but yr 11 and 12, then masks were made mandatory outside the home, but it was not until schools were closed for everyone and we entered stage 4 with a 5km radius and a curfew that numbers stared to come down around a week later.

I had to scrabble around a bit to find the timeline of exactly what restrictions happened when, but the actual data work was very, very simple. Yet I had not seen a graph of this type anywhere (and believe me, I was doom scrolling all of the data out there). It shocked people, because it shows really clearly that it wasn’t until we were in the very strictest of lockdowns that the numbers started to come down. Lockdown light didn’t work. Locking down individual postcodes didn’t work. In Melbourne, we are watching Sydney try all of the “can we avoid really seriously locking down” strategies that we know failed us, with a three times more contagious variant of the virus. We are like a cinema audience shouting at the screen, with as much impact on the outcome of the story.

Would our collective understanding of covid have been different if we were all more data literate? If we were used to working with real data that bounces around rather than reliably following the textbook curve? I think it would. I think if we all recognised exponential growth when we saw it, even when the numbers were small, maybe we’d be more able to resist the pressure to just “live with the virus”.

We can build data literacy by changing the way we teach (and I lay out a detailed plan for that in Raising Heretics: Teaching Kids to Change the World), but we can also do it by changing the way we communicate. Scientists, governments, people who understand data all have the power to show us the data in different ways. And no one way is going to communicate to everyone. So we all need to share our skills, tell the data stories, and show people why they matter.

The truth is, we are increasingly being faced with the kinds of scenarios where understanding data matters. From pandemics to climate change. From income inequality to fake news. If we understood more about how data works, and what it means, maybe we’d be more supportive of the scientists warning us of the brick walls we’re speeding towards.

For more reasons why we need to be data literate, and how to teach our kids to change the world, check out Raising Heretics, pre-order now, or buy online in all the usual places from August 1st.

Vaccine hesitancy is a logical consequence of the way we teach science.

Vaccine hesitancy could literally kill, yet it’s a logical consequence of the way we teach science.

We tend to think Science is about facts and right answers. This is absolutely the way we teach it, but it’s the opposite of what Science really is. We learn the periodic table, the arrangement of subatomic particles around a nucleus, the equations for force and motion, and how to name the components of a cell. We teach with experiments where known inputs are treated with a known process, producing a known outcome. Kids who don’t get the “right” answer either fake their results or copy from their neighbours. This is not an education in Science, it’s an education in confirmation bias – in seeing what we expect to see.

Science is actually a way of exploring and understanding the world, and of solving problems. By its very nature science deals with uncertainty, and constantly proves itself wrong as new information becomes available.

Scientific theories are based on the information we have right now. Sometimes we can’t see, measure, or understand enough to explain a phenomenon fully, but we have a model we think is right, and it’s right enough to help us understand some parts of the way the world behaves.

We can see this in the way our understanding of covid19 has evolved. At first we thought it was transmitted by droplets, so that unless you were in the direct path of someone’s sneeze or cough, the main risk was touching an infected surface. As we learned more, our understanding developed. We now know that it is very easily transmitted by aerosols – in other words, virus particles can hang in the air in such quantities that we easily breathe them in and become sick.
This explains why transmission rarely happens outdoors, and why ventilation is key when we’re indoors. It also explains why hotel quarantine is so problematic – because even if there is no air transfer between rooms, an infected person walking through a corridor can leave that corridor so contaminated that it’s infectious for some time afterwards. It also explains why masks (when properly worn) are so effective at preventing transmission.

We’ve also seen our understanding of vaccines and their side effects evolve. And the fact that the story keeps changing – from “Astrazeneca is safe for everyone” to “it’s safe for everyone over 50” and now “it’s safe for everyone over 60” – makes people nervous. But it’s this rapidly changing information that should give us comfort and confidence. This is Science doing its job – adapting our understanding according to new information.

I once interviewed Cameron Neil, who at the time was head of the Fair Trade Association of Australia and New Zealand. We were talking about the fact that it’s hard to buy ethically, because the information we have keeps changing. Neil’s response is an ethical approach to consumption, but it also encapsulates an intelligent approach to Science: “With the information available to me today I make the best choice I can, knowing full well that I may get information tomorrow that means the choice I made was the wrong one, and I’ll have to do better next time.”

When it comes to vaccines, of course, we crave certainty. No-one wants to take something that might harm them. We want to know with absolute clarity what the best thing is that we can do for our health. The fear and uncertainty in the community around the Astrazeneca vaccine is palpable. Yet according to Hassan Vally, an epidemiologist at La Trobe University, the risk of dying from a blood clot due to the vaccine in Australia is 0.5 per million, while the risk of dying in a car accident in Australia in any given year is 28 per million. Compare that with the risk associated with taking aspirin or other non steroidal anti-inflammatories (NSAIDS), which is 24.8 deaths per million people, or a staggering 153 per million users of those drugs. This is where a different understanding of Science could help us.

If we truly understood how Science worked, the rapidly changing information would give us confidence that our understanding was getting better and better. If we taught Science as an exploration of the unknown, and a constantly developing set of theories, rather than a fixed set of hard facts, we would be far better prepared to understand the constantly evolving picture of covid19 and its vaccines.

It’s really hard to teach kids critical thinking skills when your toolkit is questions that all have right answers, curricula full of facts and straightforward procedures, and textbooks that leave kids floating on an uneasy sea of factoids, memorisation, and perfectly neat examples tied up with a bow.

Imagine if we taught Science by exploring the world. By trying to solve problems that have no textbook answers, where students have to rigorously test and evaluate their own work (and the work of others) in order to be confident of their results, because they can’t just look up the answer, and have the teacher mark it right or wrong.

If we grew up with this basis, knowing Science as an evolving, developing discipline, rather than a bunch of facts pinned to the unchanging pages of a textbook, we would know that our changing understanding of covid19 and its vaccines is not a threat. It’s what’s keeping us safe.

Read more about the way we teach science, and about teaching our kids to be critical thinkers, in Raising Heretics: Teaching Kids to Change the World.

Teaching STEM is more important than robots

This is an excerpt from Raising Heretics: Teaching Kids to Change the World, which is due out on August 1st.

I founded the Australian Data Science Education Institute in 2018 because I wanted to show kids that they are capable of working with technology, that it is relevant to them, and that they don’t have to look like Sheldon from the Big Bang Theory in order to learn to program.

It’s well known that the technology industry has a diversity problem when it comes to women, but lack of diversity goes way beyond gender. By trying to increase the number of women and girls in STEM, we are only tackling the easy part – though it’s actually not that easy, judging by the sheer volume of women in STEM programmes and the persistently stubborn failure of the numbers to actually shift.

The problem is that we consistently attract the kinds of people to tech that are already there. We are missing big chunks of the population – boys included. Boys who don’t see themselves as nerdy, or who don’t see the point of tech. Girls who don’t see it as relevant to them. Non binary and gender queer kids who don’t see themselves as represented or welcome in any of the tech programmes available to them.

If we had true diversity in technology and Data Science, we’d have a range of ethnic and cultural backgrounds, as well as people with a wide range of physical abilities. We’d have people on our design teams that are mobility compromised, vision impaired, with allergies, with varied gender identities and sexualities, with every possible skin tone and body shape. We’d have people who act differently, dress differently, think differently, and have different needs. I have headphones that don’t work well with long hair, for goodness’ sake! Guess who was on that design team?

This lack of diversity is bad for the technology industry, but it’s even worse for the rest of us, because technology is changing the shape of our world at an alarming rate, and we currently have very little say in our own future. Companies like Uber and Doordash are radically changing our working conditions and eliminating hard won entitlements and protections, while Facebook and Youtube spread misinformation and encourage radicalisation, all in the name of keeping people on their platforms and maximising their profits. Our world is being directly shaped by technology companies that are working in ways we don’t understand and have no control over.

Meanwhile we see human resources companies using AI to filter job applicants, claiming that their system eliminates “human bias”, without admitting the possibility that it introduces new forms of machine bias. We see “predictive policing” algorithms being used to predict crime and target particular communities in disturbing ways. We see a rush towards machine learning and artificial intelligence systems for their own sake, rather than for problems they can legitimately solve, and we have a wholly unwarranted confidence in the accuracy, reliability, and objectivity of their output.

It turns out that diversity in the technology industry is only a small part of the reason why teaching all kids Data Science and STEM skills matters. The big part is that we need a technology and data literate population who are trained to think critically and creatively, and, in particular, trained to believe that they can solve problems. That’s the world we need to build. And the foundation stone of world building has to be education.

We have a choice. We can train kids to be obedient process followers who don’t rock the boat, or we can train them to be challenging, critical and creative thinkers who ask difficult questions and come up with innovative solutions to our worst problems.

Above all, we need people who are prepared to be heretical.
Who ask “why?”
Who ask “how can we be sure?”
Who ask “what have we missed?”
Who ask “how can we do better?”
Who ask “who are we hurting?”
Who ask “how can we fix this for everyone?”
Who ask “how will we know how well it works?”

These questions are often heretical. By asking them, I’ve sometimes made my bosses very unhappy. They make people uncomfortable. But they are crucial to building an ethical, sustainable, positive future for all of us.

___

Pre-order Raising Heretics now:

https://events.humanitix.com/raising-heretics-pre-orders-and-book-launch