Amazing conversation with Dr Emily Kahl from Pawsey Supercomputing Research Centre, on everything from the need for a humanities education in STEM, the application of Marxist and Feminist Lenses to Data Science, and a whole lot more. This was an absolute delight. Check it out!
“I wanted to use the tools of maths and science and the life of the mind and thinking about things rigorously to make things better for people and to help people out. And I think, you know, there’s a lot of, a lot of forces arrayed against us that make it, you know, difficult to do that. But we’ve still got to try.”
“we’ve made such tremendous, tremendous strides in making the world better for people and really blunting some of the worst parts of, you know, danger and disaster and human cruelty. And that is still really exciting to me.”
“is there even something to explain here, right? Or are we just fitting to the noise? Is there actually a pattern to be found here? And that’s again where you sort of need to step back from the engineering process of how do we tune all the parameters? How do we fit in more data? and actually think like what are we trying to do here?”
” it’s fancy curve fitting, right? You’ve got at heart a neural network is some kind of universal function approximator. You feed it a bunch of data on the assumption that there is some complicated high dimensional curve that explains that data and it comes up with in the limit an arbitrarily good approximation to that curve. There’s a lot of assumptions baked into that. ”
TRANSCRIPT
Linda: Welcome back to another episode of Make Me Data Literate. We’re having a bit of a Pawsey supercomputing center theme running at the moment because they do so much amazing and fascinating science. So I’m very pleased to introduce to you Dr Emily Kahl. Welcome Emily. Thanks for coming.
Emily: Thanks Linda. It’s great to be here.
Linda: Excellent. So we always start off with who are you and what do you do?
Emily: So my name is Emily Kahl and I work at the Pawsey Supercomputing Research Center and I get supercomputers to simulate chemical reactions. I teach computers how to do chemistry.
That’s very cool. I love the variety of stuff that happens at Pawsey.
Emily: I have a non-facetious answer as well to what I do.
Linda: Go for it.
Emily: Yeah. I mean, what I do is we have a wide variety of scientists who use our machines, but having the actual physical hardware – We run a very large supercomputer at Pawsey for those who don’t know. That’s the largest in the Southern Hemisphere called Setonix. But it’s no good to have hardware if you don’t have working software and if you don’t have scientists who know how to use it.
So my job is to take a lot of computational chemistry code that exists out there to simulate what goes on when you have atoms and molecules in some complex chemical reaction and make it work really well on our systems and then train people up to use it effectively.
Linda: That’s very cool. I’m about to get to the education question that I always ask, but how did you wind up getting into HPC? I didn’t warn you about this question. So…
Emily: Oh, it’s a stitch up! I had a bit of a circuitous route. I didn’t originally intend to get into this field. I studied quantum physics. So I didn’t even study chemistry in university. I studied quantum physics and I did a PhD in atomic physics. And atomic physics is a really cool field because the experimental side of things is extremely precise. There’s a lot of really clever experimental techniques that mean that some of the most precise measurements that we have in physics come from atomic physics and from atomic systems, things like atomic clocks or precision atomic spectroscopy.
But the theory is really, really difficult. The mathematics involved is completely intractable with your old school, if you think of like a chalkboard or pencil and paper type, trying to find what we call an analytical solution. So something that has a finite number of terms and some algebraic expression. That’s like a sum that you can put things into. The maths is completely intractable for that for any complicated atoms. And complicated here means more than one electron, two or more electrons.
Linda: Whoah!
Emily: So yeah, exactly. We can do hydrogen and hydrogen like ions with a pencil and paper. But once you get beyond that, even helium, I did my honors thesis on photon atom scattering in helium and it turned out to be devilishly difficult to get right. So you have to do a lot of computer simulations and numerical solutions. So like number crunching and getting something that’s close enough. And during my PhD, when I was doing some of this work in complicated many electron atoms, I ran into this situation where the code that we were using to do this number crunching didn’t quite have the features that I needed it to to do my physics. And my supervisor who had written the code during his PhD, his thing was like, look, either you learn how to modify this code and you write it yourself. Or you pick a different topic to work on.
I was like, okay, well, guess I’m learning how to write code. And one of those things, I mean, that’s a little bit, you know, a little bit of a cute story. I’d already known how to write code, but I didn’t know how to do like real production stuff. And a lot of the stuff they teach you in university in a computer science degree does not really correspond to what you do out in the real world. That’s a bit of a, yeah.
Linda: So true. And I love like, I think I’m going to have to add this question in at least for anyone who’s in HPC because one day I will meet someone who who intended to go into HPC, but it hasn’t happened yet, and I’ve met a lot of people in HPC. It’s all like, well, they didn’t mean to, but this thing happened that I needed to learn to write code.
Emily: And then suddenly it’s like, oh, people will pay you to do this thing, right?
Linda: Yeah. Yeah. It fascinates me. It’s the accidental industry. And I think it’s the most accidental of any industry that I know of. Like, I just don’t know anyone who intended, who set out to go into HPC.
Emily: I’m mentoring a student right now who is interested in becoming an HPC research software engineer. She just discovered like, oh, this is really cool. You can play around with supercomputers. That sounds great. That’s what I want to do. Like, good on you, honestly. You have your life way more together than I did at that age.
Linda: That’s awesome. I wonder if it’s changing. Like, I wonder if, I mean, I hope it is changing because I do a lot of work in that space. And I know Pawsey does too and kind of trying to reach out to school students and university students and basically get HPC on the careers map. So it’s good. Hopefully it’s actually working.
Emily: Yeah, exactly. It’s a good sign.
Linda: That’s definitely a new thing. So back to the actual questions.
Emily: Yes, of course.
Linda: What did you have to learn to do your work? What was missing from your formal education? You’ll notice the assumption there is that things were missing.
Emily: Yeah. I mean, it’s tricky, right? Because the field that I work in now is not the field that I studied for. And even the field of science, ignoring the whole HPC part of learning out a right code and right efficient code that can run, you know, massive parallel scale. The field of science that I work in, of like computational chemistry, there’s some overlap with physics because at the end of the day, you’re sort of you’re still dealing with the motion of atoms and molecules, but it’s not the applications or the area or the context in which I studied.
So like a lot of what I do right now, I kind of had to learn on the fly. I had to learn on the job, right? And I suppose, you know, if we were to talk a little bit about what was missing from my formal education, there is that sense of like a more of a broad focus on just doing a little bit of everything and learning how to pick up new stuff rapidly. You know, the university in Australia in particular is moving much more towards heavy specialization, right? And I’m not, I feel like that was not a great direction for me at least because it was kind of like, I just, you don’t know where you’re going to end up, right? Very few people who study, for example, physics end up getting a job as a physics professor.
And there’s some movements to sort of address that in the sort of formal university education. And I welcome that because that would be great. That would have been great for me.
Linda: Yeah, yeah, there’s a lot we could do about university education, but that’s a topic for a whole different series of podcasts.
Emily: Yes. I will just put a pin in this because I suspect given some of the questions that you ask, I may have more to say about this as a little preview as well of the broad topic, not just within fields of science, but I’m also a big fan of humanities education. And that’s something that I sought out for myself independently of science that I think is really valuable in this field. But perhaps we can, perhaps we can circle back around, do a little conversational gamut.
Linda: It’s funny we come to that now because I have scheduled in a few weeks an interview with Mark Stickells on this podcast. And you know that Mark Stickells, the CEO of Pawsie, does not have a tech background.
Emily: No, absolutely.
Linda: He has an Honours degree in English Literature. And so we want to talk about the importance of those skills in data science and HPC and technology in general. So
Emily: I am so keen.
Linda: So be looking out for that one.
Emily: All right. I mean, let’s go. You’ve brought it up. I think, I think, let’s do it.
Linda: Let’s talk about it now!
Emily: Yeah, yeah. Like the humanities, the whole point of like the arts and the humanities, right, if I can be a little bit overly reductive is to teach you like, or for you to sort of learn and think about how do I be a human in a world of other humans? How do I navigate people and societies and the structures that we build and the knowledge that we create, right? And like science doesn’t happen in a vacuum, sure of all contexts. It’s a human endeavor fundamentally, right?
And one of the great triumphs of the humanities is this understanding that we are always going to come at problems that we address, whatever they are, coming from some particular greater context, some structure that informs some, you know, ideological or philosophical or epistemological superstructure that informs how we view the world, right? And you cannot step outside of that no matter how much you try. It is simply not possible.
And a lot of 20th century, 21st century humanities is trying to grapple with that idea of we can’t step outside of this. How do we still try and do good work, regardless of that? And that is something that I think a lot of scientists and especially a lot of engineers in the tech field, not to get too engineer bashy, but a lot of people in say data, right, will view data as this apolitical thing. It’s like, here’s the data, we’ve made a decision, that’s it. But the decision of what to collect, how to collect, how to present it, these are all going to be informed by your position in society, your upbringing, your history, your cultural context, and being self critical, and then also critically evaluating how other people view this through different lenses.
You could take a, I’m going to say a dirty word for a second, you could take a Marxist lens or a feminist lens or a racial lens and doing all of that as an attempt to try and falsify your hypothesis, your own perspective, right? This is a critical part of science that I think is not well taught in STEM and should be.
Linda: Oh, the idea that you should be trying to falsify your hypothesis is something that we have a desperately dangerous tendency to forget, that what we set out to do is prove the hypothesis and that is not science. We should be trying to disprove it as hard as we can before we say, okay, maybe this one has some validity. Not proof, because you can’t prove things, but some validity.
Emily: But the more you attack it, the more you can be sure that there’s something there, right?
Linda: Right, you are so singing my song. And just to come back to that idea of data, saying what you collect and how you collect it, the definitions are always so interesting to me that define a student at the school. Is it someone who’s enrolled on a particular date? Is it someone who’s there today? Is it what about kids who are sick? What about someone who’s been away for a term? It’s not… The definitions are actually complicated in almost anything that you try to study. But we have this fallacy, this absolutely ridiculous idea that when you collect data, it’s rigid and it’s objective and it’s straightforward. And you as a person, your history, your ideology, don’t enter into it. And it’s not actually possible for people to do things that their whole personality and history and ideology don’t enter into. And I think the myth that science is objective and straightforward and rigorous is actually very dangerous.
Emily: Yes, there’s a lot of epistemic snobbishness among scientists. I think the best scientists appreciate this, right? And will actually go out of their way to form this. But there is a lot of sense of like, “Ah, well, STEM is the thing that’s like, we’ve got maths behind it, we’ve got experiments, you know, therefore it’s real.” And it’s kind of, “Oh, that’s very dangerous ground to be treading on.”
Linda: Yes, I always like to, when I was doing my PhD and beyond, I used to tell people I was in the people end of computer science and they’d be like, “Computer science has a people end.” Yes, it’s where the important stuff is. Like if you take the people out of computer science, you haven’t got anything useful.
Emily: Who do you think uses this stuff? Exactly. If anyone’s ever tried to use a user interface made by a programmer, they will know the pain. If you’ve looked at the code that like, I’ve tried to write that’s been a graphical user interface. Oh my goodness me.
Linda: Shall we talk for a moment about my gov? No, let’s not. We could go forever.
Emily: No, goodness me. Exactly, abort, pull the ripcord.
Linda: Oh yes. Oh, I didn’t get, it didn’t expect to get a rant about interfaces from this podcast but I’m very here for it. Yes, no, exactly. Well, that’s what you get. I think having the human side, a lot of people talk a big game about, you know, remembering the human but it’s actually really hard. Particularly when you start thinking about the wide variety that humans come in.
Emily: Right.
Linda: Well, that’s it. Remembering the human who’s just like you, that’s easy. I can design for myself, that’s no problem. But designing for myself and for you and for my kids and for, you know, someone I’ve never met and for, you know, actually anyone who might want to use that, that’s really hard.
Emily: Yeah, the person who’s, you know, got a baby in one arm and a phone in the other and is multitasking all of these things. Like they’re going to have a completely different user experience to someone sitting at their desk with noise cancelling headphones on.
Linda: Yeah, yeah, 100%. Yeah, that was very satisfying. Excellent. Back to the questions momentarily. I suspect we’ll digress again.
Emily: Yes, leaving aside the ritual airing of grievances. Yeah. Two minutes hate.
Linda: Well, this is a good grievance question as well. Is there anything that you wish everyone knew about data? One thing that you would like if people understood this, everything would be better.
Emily: Oh my goodness me. That is such a broad, broad and good question. I think at like a very basic level, I think we could definitely benefit from everyone, everyone knowing a little bit more about statistical literacy, reasoning about things like probabilities. What does it mean when something has a 90% probability to occur? Because most people just kind of think of it as like, the probabilities are like it’ll never happen. It’s like a 50/50 or it’s guaranteed to happen. And that is a cause of much sort of mistaken conclusions or people behaving in sort of bad fashions or making poor decisions as well as things like, you know, knowing how people can use statistics and the selective application of data to mislead them. And this ties into that previous thing about different frameworks and lenses, right?
Linda: Yes.
Emily: Like thinking about who is presenting this data and what is their material interest in doing so? Like if we’re getting back to, I know I was being deliberately inflammatory by using a framework of like a Marxist framework, but I think it’s actually really useful, right? To think in terms of whether or not you agree with Marxist political end goals. To think of it as social and economic classes with key material interests. How do we then look at, let’s say someone has produced a graph in the paper, who’s done that? What’s their position within the broader hierarchy of society? And what are the goals of that class in presenting it? And just that kind of critical thinking, I think would really go a long way into helping with data literacy.
Linda: Oh, you’re not going to get any argument from me on that one. I mean, it’s a huge chunk of what I do is just trying to build that data literacy into the way we teach all of the things. And often when I see people set out to teach data science, they start with the technology. And I’m not interested in the technology. I’m sorry, the technology is the easy part. People are the hard part. Let’s start from the literacy, start from the communication, and what are we trying to do here? Why are we trying to do it? Is it a valid thing to do? What’s wrong with this data that we’ve collected? All of that kind of stuff. I think if you come out with no technological skills, but that literacy, that’s a huge win. I’m thrilled with that.
Emily: That’s really funny because people always, there’s this recurring thing of like, we should teach critical thinking in schools. And my response is like, we do. It’s called English class. You just weren’t paying attention because it didn’t look like maths and statistics and the like. But again, that sense of teaching the human aspect and the context, right? Or the lack thereof. Like so much of what can lead people to either make data mistakes and none of us are immune from this. I’ve done this myself as well. Or have deliberately misleading data is we’ve had this, we’ve got this huge context collapse with the internet and with social media and with the sort of flattening out of communications. It means that like data that has been peer reviewed and really battle tested and has gone through multiple, you know, attempts to try and address those systemic biases versus something that’s just like marketing copy look functionally the same at a first glance, right?
So you can’t just rely on something not having spelling mistakes and looking reasonably flashy and glossy and arguably you never could do that, right? But it’s especially true now. And so there’s just, there’s no shortcuts to it, right? You have to do that hard work.
Linda: Yeah. And I think when you separate the critical thinking from the maths and science, you lose something. You really, you know, you break the important part of maths and science. Maths and science should be unteachable without critical thinking. But we teach people to apply known processes and get predictable outcomes. And that’s not science. It’s not really maths.
Emily: Otherwise you’re going to fool yourself, right? Like the whole point of science is to try and find some capital T truth, right? And if you’re only, like you said, sort of the temptation is to only ever prove your own hypothesis because that feels good psychologically, but you’re going to be misleading yourself if you do that.
Linda: And the rest of the world.
Emily: Yes, exactly. I feel like we’ve been, we’ve been doing a lot of agreeing. So ask me a controversial question, something that we can really get into a barney about.
Linda: Challenge you, fight you. This is not going to be that question. What are the worst data mistakes you’ve seen?
Emily: So I don’t know if it’s the worst, but one that’s really struck out as being particularly egregious was just last week or the week before when OpenAI released a benchmark for chat GPT5, and it was essentially a press release and a marketing copy of Look How Good It Is. And they had this graph, this infamous graph where they were comparing results on some benchmark for GPT5, the new version with GPT4O with some other models. And they used a bar chart and they had a big bar for GPT5 and a small bar for GPT4O. But the GPT5 benchmark was lower. It was a smaller number, but they used a bigger rectangle to represent it.
Linda: duh… what???
Emily: This is an audio media, but you can just imagine us just like incredible. But like, it’s just the most like you would, you would give a big cross and a “see me after class” to a primary school student for like, “Hey, we need to talk about why this is bad.” And there’s a billion dollar company, multi-billion dollar company, ridiculous.
Linda: See, in an ideal world where everyone was learning the way I want them to learn, that wouldn’t work because people would know to compare the values and to look at the y-axis and all that stuff, all that good stuff. But unfortunately, it does work because we are very, we’re actually very easily persuaded in a lot of ways. And we see a graph and we see something that’s bigger and we go, “Well, that’s better than the thing that’s smaller.”
Emily; We like pictures. Monkey Brain likes pictures. It’s the problem with the internet is we spend so much of our life looking at images and so much of the world is mediated through two-dimensional images that it’s like, it’s so easy to mislead with that.
Linda: Yeah.
Emily: And I look to be charitable to Open AI. You know, this was just like, I don’t know, maybe they got chat GPT to make this graph and it hallucinated. Maybe it was just a regular howler and someone didn’t check it. But to me at least, it is the intentionality of whether they meant to do it is less interesting than the end result, which is that it was a misleading approximation or a misleading bit of presentation of data for how good it is. That serves their interest of they’re trying to sell you a product, right? So they want it to look good. And yeah.
Linda: Yeah, I think it’s charitable calling that a mistake. But you can’t know the intent.
Emily: Yeah, exactly. And so you can just look at the outcome and the outcome is, well, it was beneficial until people on Reddit started pointing it out and then Sam Altman had to do a very bashful apology. But the initial impression stuck, right? Like a lot of people saw the graph and went, “Oh, that looks cool.” And then didn’t see the correction.
Linda: Like that Pratchett line – A lie can run around the world before the truth has got its boots on. Like that lie is gone. We can’t catch that lie now.
Emily: One of the best.
Linda: Yes. So this is a related question and I guess you’ve already answered it, but I’m sure you have more examples. Have you ever seen data deliberately misused and how do we spot those things?
Emily: Yes. The easy answer is you cannot trust anything produced by someone trying to sell you something because they have a very strong incentive to present their product in the best light possible. And it sounds trite, but also it is a trap that people fall into again and again. The open AI one is one of many examples. I see it a lot when I look at results produced by, for example, computer hardware vendors for high performance computing, where they’ll have a bunch of benchmarks that say, “Hey, look at how cool our hardware is. It gives good results.” But if you actually dig into the results yourself, you’ll find that frequently either that’s only part of the story and they’ve selectively chosen benchmarks that make them look good, which is very common, or they’ve done some other chicanery where they’ve just cherry picked results or even just made their own benchmarks.
And so to give you a concrete example, because I’ve been very abstract recently, there was an example where I was writing up a paper with a colleague of mine where we were doing a benchmarking of some machine learning intratomic potentials for computational chemistry. So can I get technical for a second, Linda?
Linda: Go for it.
Emily: So the field of study in which I am sort of, I guess an expert now, I’ve been doing it for long enough, is called molecular dynamics. And it is a way of simulating the motion of molecules in some chemical or physical process. The idea is you have a bunch of particles in a box and then you simulate how they move around. They have some interactions with each other, some forces that cause them to move around, according to some equations of motion. Typically we do classical good old Newton’s equations of motion, F = MA. And we can use a field of physics called statistical mechanics to link the motion of the particles that we simulate to bulk thermodynamic properties that we can measure.
So this might be things like the viscosity of a liquid. We want to simulate some chemical lubricant, right? So we can simulate the motion of the molecules and then we can link that to what’s its lubricating behavior that we can give to experimental or chemical engineers or that sort of stuff, right?
Molecular dynamics is a really powerful tool for computational chemistry. It’s like a digital microscope my old boss likes to call it, gives you this insight into molecular behavior but it’s also really computationally intensive. You need to calculate these interactions between a lot of different particles because you know you’re doing really really small stuff, linking it to really big stuff. That’s a sort of deliberately hand-wavy approximation that’s close enough.
And one of the really interesting tools to come out of machine learning stuff is the idea that we can apply machine learning techniques which to some degree are fancy curve fitting for data to approximate, to come up with really good approximations of these interatomic forces in our simulation. And we can model complicated interactions computationally cheaply by training up some kind of artificial neural network to do this stuff. And or we can use other statistical techniques, there’s a thing called a Gaussian process that’s also you know quite interesting in this field.
But the general idea is you train up a machine-learned model and you can get much faster simulations than you would otherwise with some caveats. Caveats are, it’s very sensitive to the model, very sensitive to the choice of data that you train with. Often it generalizes poorly in configurations of your system that are outside the training data that you’ve produced. And there is a whole zoo of these interatomic potentials out there that people have made.
And like I said, to a first approximation, don’t trust results from vendors. So there’s all these papers out there saying look at how cool our thing is and no one can actually or could actually give you a good answer of which one should I use. Like I’m a scientist, I just want to do science, what do I do?
So working for a supercomputing center and the fact that my job is not tied to paper publications. It’s like right, we can do this, let’s actually go and review this. So I was working with another colleague at a different supercomputing center and we published this paper and in the process of reviewing all of these things, we found so many examples of people doing little bits of, for being very generous, sleight of hand with their benchmarks. And a concrete example that I shan’t name names, they had a Jupiter notebook. So those who don’t know, this is a way of presenting Python code interleaved with some really nice data presentation and you can do interactive demonstrations of your Python code in a web browser. It’s a really cool pedagogical technique.
So they had a Jupiter notebook demonstrating their thing and we downloaded the notebook off their Google Colab because you know, it’s like it’s good to run it locally and play around with this stuff. And we simply could not reproduce their stated results.
Linda: Oh wow.
Emily: yeah, right? This is real bad. So we’re playing around with this being like, why aren’t we getting this? It’s just not converging to the same level of accuracy they’ve got. What am I doing wrong? So my colleague who is a lot more confrontational, I suppose, am I allowed to swear on this podcast?
Linda: Absolutely. This is adults.
Emily: Does not give a fuck in the slightest. He just like corners the developer of this at a conference and goes, what is going on? Like basically puts a laptop down and is like, explain yourself. Why can’t we reproduce this? And the guy very shame-facedly was like, well, actually, we did the training on a much larger model. And then we just reuse the outputs in the Jupiter notebook. And then we changed the results to a smaller one so that it would run in a smaller period of time for live demos.
Linda: Wow.
Emily: Yeah. Wow. That’s some actively, actively bad practice. Like that’s I would call that outright malicious.
Linda: Look, I mean, it’s a reasonable thing to do IF you admit to it.
Emily: Yes!
Linda: You know, like you have to say, this is what we did here. You can’t just go, oh, look, this is, you know, a bit of magic that works. And actually, it’s not magic because of all the stuff that we did to make it work. Just admit to the stuff that you did to make it work.
Emily: Yes. Exactly. You can have a little thing that say, hey, we’ve gone a small one here for the sake of getting it running in a reasonable length of time. If you want to reproduce these results, run with these parameters: warning, it will take you six hours to complete. That’s all you have to do, right? And then nerds like me can go and run it for six hours and go, yes, excellent. Good job.
Linda: Oh, wow. That’s atrocious.
Emily: Yeah. Deeply frustrating. And, you know, it’s really solidified by being like, yeah, okay, we need someone who has got no skin in the game to come and actually do a proper review of these things to cut through it all.
Linda: And that’s the thing with AI. Like, I’m writing a teacher’s guide to AI at the moment. And it constantly butting up against the, well, the company, whether it’s open AI or Anthropic or Google or Microsoft, doesn’t matter. The company says that their system is this, that or the other. You know, it’s good on this benchmark. It’s that, it’s that. And I’m like, I really would like to see a lot more systematic, rigorous, objective evaluation. And, you know, they’re constantly tuning them to the benchmarks anyway. So if you’ve got it, if you’ve got a published benchmark, then the system may operate brilliantly on that benchmark, but not be able to tell you the two plus two is four. And the press just breathlessly reports the marketing copy from the companies. You know, I’m writing a keynote at the moment for November and I wanted like screenshots of talking about how Sam Otman claimed it was PhD level intelligence GPT five. It was like, that’s insulting to those with PhDs. But anyway, it’s insulting to people actually never mind whether you have a PhD or not. But, but I just, I, I Googled, you know, PhD level GPT five. And I just got headline after headline after headline, zero skepticism, zero analysis, just chat GPT five has PhD level intelligence. And sometimes it said Altman says, but like, that was the level of, you know, like, how about, you know, a different headline like Open AI makes outrageous claims for it’s latest model. How do they stack up? Yes, exactly. Hard to do. You know, you don’t actually need
Emily: Explain the journalism!
Linda: You don’t need a PhD machine learning to, to compare it, you know, to, to pull up five and four and go, you know, I want to ask it these questions and see what it takes so little time to hit up on things that it doesn’t can’t do and that it gives nonsensical replies to. And
Emily: yes. And if you dig into it deeper and look at what kind of benchmarks are they focusing on, oftentimes they’re things that, like you said, are finally tuned to make them look good, or they tune the model to the benchmark. And I don’t know quite what the answer to that is aside from this almost like, we just need an adversarial constantly coming up with new things to stress them at, which I guess we kind of do emergently in the form of people on Reddit or mastodon on or whatever, who are curmudgeons like me being like, Hey, I asked it to count how many US states have an R in the name and it told me Mississippi.
Linda: Yep. Yeah. But the problem is like, none of it seems to matter because like they don’t know. It doesn’t come through. No one pays attention to this. It’s wild. It’s wild. You know, there are still people using it to for serious purposes and trusting the outputs. I used in a recent blog post, I used the example of a friend of ours who we caught up with in London who looked up restaurants that have good gluten free options because I’m celiac and he found one and got, you know, chat GPT or whichever chatbot he used said it was wonderful and we got there and they literally could not feed me anything. They did not have any gluten free items on the menu. The menu literally said we don’t feed celiacs.
Emily: So much of it is just people trying to apply these things outside of areas where it makes sense, right? Yes.
They’re so useful in their little appropriate niches. But
Emily: yes. And again, coming back to humanities, right? Like I’m not a expert in linguistics or natural language processing or anything like that. I gather there are actually legitimate useful things that you can do these for frequently as a front end to some other more deterministic and applicable backend. But you know, I come back to this perhaps overly reductive and certainly the AI boosters would say that it’s overly reductive. But this overly reductive view of it’s fancy curve fitting, right? You’ve got at heart a neural network is some kind of universal function approximator. You feed it a bunch of data on the assumption that there is some complicated high dimensional curve that explains that data and it comes up with in the limit an arbitrarily good approximation to that curve. There’s a lot of assumptions baked into that.
And things like is there even something to explain here, right? Or are we just fitting to the noise? Is there actually a pattern to be found here? And that’s again where you sort of need to step back from the engineering process of how do we tune all the parameters? How do we fit in more data? and actually think like what are we trying to do here? You know, how does this work for science, for chemistry?
Like large language models, I don’t think they’re useful in the slightest, but taking some of the tools like transformers, for example, which underpin this technology or some of these different, you know, neural network encodings that are beneficial for say image generation, your sort of diffusion models, those can be really handy for approximating this because there is a well-defined physical thing. It might be hard to calculate, but it’s there, right?
Linda: Yep.
Emily: And I have this thing that is just kind of like, how can we get that through to people? How can we how can we educate the average punter? Because the AI companies sure as hell aren’t doing it because they want to make money from this.
Linda: They have a vested interest in the average punter being highly uneducated.
Emily: And that’s nothing against the average punter, right? Like people got stuff to do, right?
Linda: And also we’re all average on some topics. None of us are expert on the expert level on everything.
Emily: You can’t. It’s not possible.
Linda: Yeah. I always say with data that data can tell you what it can’t tell you why. And AI often is not even telling you the right what you’re or a you know, it’s created a what out of nothing. But it definitely can’t tell you the why. And that’s really important. That’s where you need the humanities and the thinking, why are we doing this? And what does it actually mean? You know, “correlation is not causation” is like the antithesis of AI. AI is like every correlation is causative.
Emily: It’s a Paradolia machine. It’s just for finding patterns, right? Yes.
Linda: Yeah, exactly. Yeah. This is a very satisfying rant.
Emily: It is. It is. And I mean, part of the thing as well is just it’s shown me as well, not that, you know, this is a new observation, but it really underscores just how much a lot of people or how little I should say a lot of people value things like art and writing and the human process and human connection. Because people will go, hey, look at this output from say chat GPT. This could replace the average, you know, this could replace some technical writer or some creative writer. Why would I pay an artist when I can get stable diffusion to generate it?
And you look at it and you’re like, well, for starters, this is mid at best, but also this is completely misunderstanding why we do any of this as people as well. And like, it’s not to get too political on this podcast. But the whole point of I think a lot of this is we have this managerial class who really hate the fact that there’s human autonomy and especially in novel knowledge industries, where we have these highly trained workers who have a lot of autonomy and a lot of power and a lot of cultural capital. And it’s hard not to look at the big push to replace all these different knowledge workers with AI and then eventually down to, you know, manual laborers as well and people who are doing just as technical and difficult work but, you know, using their hands rather than sitting at a desk. They’re coming for that as well as a way for the people who currently run the joint to be like, great, now we don’t have to pay anyone. And now we don’t have those workers demanding that we behave in an ethical fashion and all of that sort of stuff.
Linda: Just AI is not going to unionize anytime soon.
Emily: Yes, it’s just purely a way to further devalue this kind of human connection. And I don’t understand that mindset. It’s fundamentally antithetical to me. But you’ve got to find some way to deal with it as human appreciators.
Linda: I’m very here for politics on this podcast. I mean, I don’t think you can discuss technology without getting political unless you’re really not getting into the meaning of it and the value of it. And, you know, it’s the definitions.
Emily: It’s reshaping our society.
Linda: Yeah. And the definitions we were talking about before, the definitions of the data that we collect and how we define things in particular categories, that is extremely political.
Emily: Yeah, I’m old enough to remember when the big furor around machine learning stuff was that the data being collected was not representative. And that’s still an issue. Facial recognition technology being very bad for ethnic minorities. Chat GPT being really bad at issues that primarily affect women because women’s data has been undervalued. That’s where looking at things through a feminist lens can be really useful, right?
Linda: Yeah. 100%. What’s the first question you ask when you look at graphs in the media?
Emily; I feel like we’re sort of, yeah, we’ve been sort of dancing around a little bit. I’d rapid fire would be, you know, who’s producing this? Why are they producing this? What’s the story they’re trying to tell? I think in terms of stories, most humans do. And then you can use that to refine your questions. There’s a lot of technical questions you can ask that open AI thing about the bar size. That’s an obvious howler, but sometimes people will have less obvious mistakes where just the relative sizes of the bars are wrong and that can be misleading. But there’s all sorts of little niggling technical questions that you can get into. But really the ultimate thing is what is the context in which this is being created and how did it get created, right? And for what end to what end?
Linda: I like that focus on stories. And I think that that comes back to your question about how do we tackle this, this problem with people’s understanding of AI, that’s got to be stories too. Yeah. And I’m actually doing a writing masterclass at the moment. And part of the reason I’m doing that class is to level up my writing in terms of stories. You know, I can do writing, explaining technical stuff, but I’ve got to get better at telling stories if I want this message to really cut through.
Emily: It’s important. Humans think in stories, right?
Linda: Yeah. And we’re persuaded by stories and we respond to stories and we react, you know, our emotions are engaged with stories. And that’s what you have to do if you want to. Powerful tool. Yeah.
Emily: What’s your favourite answer you’ve gotten to this question? So I feel like there’d be a wide variety, right?
Linda: Yeah, there’s a huge variety. And I think the ones that come down to stories are, you know, as you say, there are a whole lot of technical things that, you know, the, what, where does the Y axis start as, you know, the classic, does it start zero. Yeah. I saw a graph of something. I don’t remember what it was, but something that I was like, oh man, there’s been this massive change. Oh yeah, somebody’s, the change in capacity of their powerwall battery that they have in their house, you know, battery for their solar system and the change in capacity over time. I was like, oh my God, that’s degraded so badly, but it actually, the Y axis was from like 10 to 13. So.
Emily: Oh, dear.
Linda: Ah, actually, if you, if you start that from zero, that’s, that’s a much flatter line, much less precipitous drop off. And that would have been what came out of the software, you know, that’s often the software defaults to that and, and it rescales, you know, from one graph to the next as well. It’s a frustrating thing I find with my fitness apps that they, they rescale the graph according to, you know, the current level of data. So I compare this week with last week, it looks the same, but oh no, last week was actually much lower and it rescaled because.
Emily: Yeah.
Linda: You know, and it’s frustrating because you can’t compare then and there have been some big mistakes for that reason. But yeah, I think the, the graphs in the media one is the, the two, I think best answers are stories and also, where is this coming from? Like who’s telling me this story and what have they got to gain? Is. Really important.
Emily: Yeah. And I think if you’re a little bit more, you know, messy, or you know, a little bit more about statistics, you know, one of the important things that I try to do as well is, okay, so given the story they’re trying to tell, is this even the right quantity? And a really common one is, like you said with the, you know, having a relatively small, having a small absolute change, but having it look like a big relative change because of where you started. Should we be looking at an absolute change versus a relative change?
It’s very easy to spin something that looks like a big scary number. But then you think about it in relative terms, if someone’s talking about the federal government is spending X million dollars on something, I like to try and think of it as divided by the population of Australia. How much per person is that? A million dollars sounds like a lot to you and me, but on the scale of a nation, it’s trivial, right? It’s, it’s peanuts.
And the other big one that I like to think about is levels versus rates of change, you know, is someone looking at the absolute quantity of this thing when really you should be looking at how’s it changing over time.
Linda: Yes.
Emily: And we saw this a lot in COVID where it’s people, it’s really hard to intuitively reason about exponential growth, right? Yes. And so things can look, oh, this is like a small number of cases, but like if you think about an exponential quantity, right, a small number very quickly becomes a really big number and keeps growing, right?
Linda: Yes. Yeah, we have two, we have four, we have eight, we have 16, none of those are scary numbers. No, but the pattern, the pattern there is terrifying.
Emily: The old fable about the wise man with the emperor where he says, you know, he’s got some great deed and he says the rice on the on the checkerboard, right? One on the first square, two on the second, four on the third, etc.
Linda: Yes.
Emily: It’s, yeah, exponential growth, right?
Linda: Yeah, yeah. We’re actually quite good at seeing patterns, but we don’t necessarily know the patterns to look for. And I think, you know, that that comes down to training. And if we taught these things better, I mean, actually, exponential growth is in the curriculum and has been for a really long time. We’ve all learned it. Well, we’ve all seen it, might be a better way to put it. But we don’t, COVID made it very clear, we didn’t understand it at all.
Emily: Yes, exactly.
Linda: And so, you know, that, that conversation around, well, what does it actually mean? And if we continue along this path, where is that going to lead us?
Emily: So that’s the question for you as an educator, because frequently
Linda: AAH! That’s not how this podcast works.
Emily: Too bad, it is now. So, I feel like with this whole thing of like, we do teach exponential growth, right? I remember in my maths B class in Queensland. So this was, I believe before we had the national curriculum, this was our sort of mid level senior maths, right? Yet your extension maths, which was math C and maths B was kind of, everyone did either maths A or B, and B was the slightly more difficult one. We looked at interest rates and finance and loans and all of that sort of stuff in the context of home loans to teach about exponential growth and differential equations and that sort of stuff. Fantastic unit. Most people were not paying attention in it. And this, I think, is something that to me seems like a real difficult problem to overcome, is a lot of stuff that is important as an adult, if you try to teach it in high school, for example, a lot of the teenagers, they’re not going to care. How do we, what do we do about that?
Linda: Oh, it’s all about making it real. You’re teaching in the context of home loans. Show me a 17 year old who knows anything about home loans or knows or cares or these days even believes that they’ll ever have a home loan. It’s just not, it’s not, it’s just not on the radar. But if you’ve got, to some extent, it’s about what’s happening now.
So if you can make it like topical and relevant. So when we were experiencing a pandemic, teaching exponential growth in the context of, well, you know, if you have 100 people in your year level at school and one person gets sick and your, your growth rate looks like, you know, one person gets two people sick, then what does that, what does that look like and how quickly does it look like that and put it in the context of real people? Well, your class is out within this, this much time and then your, you know, the class next to you and, you know, actually putting it in the context of something meaningful.
You know, I was talking about probability the other day and, oh man, I was looking at the math curriculum, the senior math curriculum and all of the examples I could find for math methods on probability were balls in urns
Emily: Because they can’t do gambling. Yeah, everyone knows that the only real use of probability anyone’s ever found is winning at poker.
Linda: Well, what about what’s the probability of getting through to the phone line when you’re trying to get tickets to see Taylor Swift? Okay. Suddenly, I like that one. You know, you may have to change from Taylor Swift to, you know, Baker Boy or, you know, it doesn’t matter, right? It’s, it’s,
Emily: see what’s going down at the local pub.
Linda: It’s meaningful, right? Yes. But, but also to some extent, the topic isn’t hugely important. To some extent, it just matters that they see that it’s real and that they can apply it to things and it’s not easy to see how you apply balls and urns to real stuff and also it’s not that easy to see that you, you know, how to apply home loan stuff.
Emily: Yeah, not when you’re 17 years old.
Linda: Yeah, but, but apply it to things that actually matter to them. Yeah. That’s, that’s step one. Things that they can see, right? Oh, you know, if you’re talking about Taylor Swift tickets, well, I can apply that to Chapel Roan tickets or, you know, they, so they can,
Emily: some other rare limited resource.
Linda: Yeah, they can easily map it onto something that they care about.
Emily: Yeah.
Linda: You know, if there are this many phone lines and this many people aiming for the phone lines and, you know, like
Emily: yeah, exactly. Then link it into rare events and that sort of stuff. If you want to be a nerd, you do poisson distribution statistics and stuff.
Linda: Right. The other day, a meteorite was seen over Victoria and now people are looking for, they think it, they think a part of it landed and so they’re looking for the chunk, right? Or the chunks.
Emily: That’s clever. Yeah, yeah.
Linda: Like, if that’s just happened in your area, I could write a lesson around that. And, you know, with all of the maths that’s already in the curriculum, it’s not anything particularly new, but, you know, like, what are the odds if, you know, if the odds of it coming down in this area or this, what are the odds of, you know, like, it’s, it’s make it, the kids who, who saw it or whose dad saw it or, you know, their mom saw it on her way to whatever, like, just suddenly that’s something they can engage with.
If there are bushfires in the area, you know, you talk about the maths of bushfires and the likelihood of, you know, this place surviving and what are the factors, you know, just make it real.
Emily: My favourite example that I had of something like that was when I was teaching physics, doing physics science communication during my PhD and we went out to Albury Wodonga to give a little, little science demo and I was teaching about electrostatics and every kid there knew about electric fences. This is a bush school. Right, right, cool. There’s my in.
Linda: Yep, yep, yep, yep. It’s meaningful because it’s something in their context.
Emily: But you know, if you’re in the city…
Linda: Yeah, I mean, I don’t know when I last saw an electric fence. So, yeah,
Emily: exactly.
Linda: It wouldn’t work so much for me and the suburbs. It’s just something that they, that they understand as real and, and meaningful. I’m actually in the process of developing a science curriculum that is built up from things like that. So if you’re teaching kinematics, you’re, you know, looking at the stopping distance of cars and the, you know, what differences does it make if they’re different surfaces and if it’s wet and if it’s gravel and the speed and the weight and the, you know, all of that stuff, which is meaningful and relevant.
And yeah, then you’re teaching exponential growth in the context of disease, the spread of disease and, you know, you’re teaching chemistry in the context of maybe nutrition or, you know, chemicals.
Emily; Swimming pools.
Linda: Yeah, swimming pools or, you know, what happens if you’re using bleach and you’re scrubbing the shower?
Emily: Stains out of your clothes, yeah.
Linda: How does it interact with this kind of substance? How does it impact with what if you spill, you know, yeah, exactly. Just, just those conversations around things that it would be better if people knew these things. Can we start science from that? You don’t have to finish science there. I’m very happy for you to teach about, you know, the different models of the atom, but don’t start from the different models of the atom.
Start from things that make sense to kids and that are meaningful and relevant and that they, that is useful for them to know. And then you grab their interest and they, you know, can expand out and do, oh, why is that? And what else can I learn about that? You know, you don’t, you don’t start from tedious theory. You start from, how is this relevant to me?
Emily: Yes, exactly. Ah, that’s fantastic. No, thank you for… I enjoyed this.
Linda: You may have pushed a button there.
Emily: Yes, no, I, why do you think I asked the question? I could sense that some deeper, deeper stuff, I’ll hand you the conch back now.
Linda: I have one last question for you, which is actually my favorite. What excites you about data?
Emily: Despite all of the grumbling and moaning that I’ve done in all of this, I still think that science, of which, you know, data is part of it, can make the world better. That’s why I got into this in the first place. I wanted to use the tools of maths and science and the life of the mind and thinking about things rigorously to make things better for people and to help people out. And I think, you know, there’s a lot of, a lot of forces arrayed against us that make it, you know, difficult to do that. But we’ve still got to try. That’s not a reason to throw the baby out with the bathwater.
And what excites me about it is the fact that through even going back to really old applications of data, like looking at, you know, John Snow plotting, you know, cases of cholera throughout London and finding that there was one pump that was spreading cholera to people, takes it out, suddenly saves a whole bunch of lives. All of this stuff, you know, we’ve made such tremendous, tremendous strides in making the world better for people and really blunting some of the worst parts of, you know, danger and disaster and human cruelty. And that is still really exciting to me. I’m still a, still despite it all a true believer in, in this potential to make things better.
Linda: That’s fantastic. Thank you so much. This has been so much fun.
Emily: Thanks, Linda. Yeah, it’s been fantastic conversation. I loved it.
Linda: I did not predict Marxism and any of the things we talked about.
Emily: that’s the Emily Kahl experience. I love it. We’ll have to do it, do it again sometime.
Linda: Absolutely. Yeah, see you next time.
