In this episode of Make Me Data Literate we are joined by Joshua Nibbs, security consultant and white hat hacker. Josh talks with us about how the development of AI has changed cybersecurity – for the better and the worse – and explains just how much data is being collected on us every single day.
“And then the AI has, you know, all the time in the world. It doesn’t eat, it doesn’t sleep, it it thinks faster than humans, it’s extremely good at binary exploitation compared to humans.”
“The way your mouse moves around the screen is you know, directly tied to your body’s morphology and your your brain to hand connection. So we can fingerprint how your mouse is moving on the page and then we can, you know, backtrack and have some data as to who you might be.”
“If it’s something you even remotely resonate with, then it it just goes as a little like tick that affirms what I believe and move on. Like I’m now more deeply rooted in my established belief. Whether the belief is based in reality or not, like it doesn’t really matter. Like maybe your belief is sound and just and know, backed by science. But it doesn’t matter because you’ve just consumed something that’s affirmed your belief without any reason to do that.”
“Where it’s like the more information someone has over you, the more leverage they have over you. So just I I think it’s reasonable in a a world or a mindset where you respect sort of freedoms and like rights of human beings to try and reduce the amount of leverage that people have over one another.”
Linda (00:00)
Welcome back to another episode of Make Me Data Literate. This is an interesting one and a timely one given everything that’s going on in the world of cybersecurity. And I think this is my Make Me Data Literate’s first foray into cybersecurity and also only the second time I have had a former student of mine on the podcast. So really excited about this one. Welcome Josh Nibbs.
Josh (00:22)
Hi, how’s it going? Hey, I’m excited to I’m excited to have a chat about it. It is it’s very timely with what’s happened in the past couple of days.
Linda (00:30)
It really is. All right. So can you tell us who are you and what do you do?
Josh (00:35)
Yeah, so like Linda said, my name’s Josh. I’m a former student. and I’m currently a security consultant at Tanto Security, based out of like Melbourne and Sydney, doing mostly penetration testing. So someone will come to us with an app or a computer system or you know, a huge network of computer systems and they’ll say, you know, look, can you get in? Can you access some data that you shouldn’t be able to?
Can you modify it? Do you think how sensitive is that data? outside of that, I do something called bug bounty, which is companies will put up basically a a ticket online saying, you know, these systems are fair game as long as you follow X, Y, and Z rules, and we’ll we’ll pay you a bounty if you can access this this kind of thing, do this kind of thing, usually related to PII is what we call a lot. I don’t know how often use that expression on the podcast, but sort of personally identifiable. Yeah. So
Linda (01:40)
I know what it is, but you- Perfect. Yeah.
Josh (01:44)
that sort of personally identifiable information. So some sort of, you know, email, name, phone number, address is sort of the basics, and then going up from there, so medical information, payment card information, so PCI, like credit card information.
And then yeah, I do some software as well, with a company called Cyberglass.
Linda (02:10)
Do you eat and sleep in there at all?
Josh (02:13)
I try my best. I I eat at my desk a
Linda (02:16)
Sounds like you
Josh (02:16)
lot.
Linda (02:18)
you’re living a lot of a lot of lives there, not just the one.
Josh (02:22)
Yeah. Well it’s all it’s all computers, right? It’s all all day, every day, hands on keyboard. I try to get out and do some bouldering sometimes.
Linda (02:33)
Nice. That’s fun. So, what did you have to learn to do your work? Was was there anything missing from your formal education?
Josh (02:45)
There’s definitely a lot missing. Obviously throughout school I tried to stay close to computers and moving on to tertiary education I went and did computer science as sort of the natural next step. a lot of what they teach you there is obviously quite academic rather than practical and it’s also primarily focused on sort of, I would say, building things rather than breaking things. And in my industry in particular, we’re definitely, you know, there’s some creation to it. There’s a creative aspect in like what you’re doing. But you are typically arriving to something that’s already been made and trying to do something that it shouldn’t do. You’re trying to convince a computer or a system to do something it wasn’t designed to do. And typically the formal education is designing systems to do what you tell them to do, and then finding out where you were wrong in your description.
So it it the the formal education is definitely lacking. I know there are some courses coming out in unis. I think they’re established now, like cybersecurity bachelor degrees and things like this. There are there are tons of like certifications and stuff you can get that are more relevant than any sort of university degree.
Linda (04:12)
So I know from we were chatting recently and you were saying that AI has significantly changed the landscape and really quite recently and dramatically. Can you tell us how AI has impacted your work in cybersecurity?
Josh (04:30)
Yeah, so especially past couple days, just I to timestamp it, this is I think it was the day before yesterday, PHP2shell came out, which is sort of the two big vulnerabilities recently discovered in PHP itself, which is pretty bad, pretty up there. And these things do happen, right? So you might remember Log4j was a big one.
A few years ago. that was related to Java applications and the way they did logging. And that basically gave attackers, adversaries, like access to thousands of computers around the world, sort of in innumerable numbers of systems, because er this is how people who wrote Java applications were logging their errors and things like that. So everyone’s using it and it was something as simple as sending a message to the server like, Hey, I want to run this command, and it would just happen.
Linda (05:31)
Mm-hmm.
Josh (05:31)
That was quite easy even then for someone to exploit. Moving forward to sort of 2021, there was sort of a bug with sudo, which is the the privilege step up mechanism in Linux. And so I think it was a heap buffer overflow or something like that. So a binary exploitation bug, it it requires like quite a lot of expertise to exploit. And even then someone put out an extremely detailed technical write-up on how it worked, how to cause the unwanted behavior, and it still took, you know, the best in the world three days to come up with something.
What we just saw today with PHP2shell, so five years later now, it’s 2026, is there was a leak the other day. I I think it was about a day before the embargo on the vulnerability came out. Usually people keep these things secret so that the defenders get a bit of a head start. I think what happened is some one of the defenders pushed their defenses out to the internet before everyone had agreed to, and people started working backwards from there. And then it got very dramatic and things sort of spiraled out of control very quickly.
But from the extremely limited amount of information people are able to replicate the bug and I would I wouldn’t call it like fully weaponised, but exploitable, usable, impactful in under twenty four hours. So it’s at the very least we’ve gone from, you know, three days with an extremely technical write up giving you exact details, down to twenty four hours with vague hints at where it might be and what it might do.
Linda (07:25)
And why did AI change that? Like what what is it about AI that meant that suddenly you could do that in twenty four hours with very little information to work on?
Josh (07:36)
Yeah, so I what I what I’ve experienced with AI and what I’ve heard from some other people is that it’s extremely good when you give it sort of a large search space problem with an extremely concrete outcome. And-
Linda (07:52)
Mm-hmm.
Josh (07:52)
hacking is basically that to a T, right? You’re saying here’s this huge computer system, all these parts, moving parts, systems and our concrete outcome is we want to run a command on it or we want to, you know, become admin when we’re not already admin, there’s a that’s a very yes or no kind of question. There’s no interpretation, there’s no judgment call for the AI to make. So, you know, some some things like art, right? If you say like paint me a masterpiece, well what’s a masterpiece?
Linda (08:23)
Mm. Mm.
Josh (08:25)
You can’t, I I mean, I’m sure people have tried, but it would be very hard to write
Linda (08:28)
Ha ha ha ha.
Josh (08:29)
down on a piece of paper where it has to use red to be a masterpiece. You you can’t say something like that. Whereas with an exploit, you can say, well, I put a secret number on this computer that you can only get if you’re the admin. And so once you have that number, you have achieved your goal. That is a yes or no. It’s a binary decision. And then the AI has, you know, all the time in the world. It doesn’t eat, it doesn’t sleep, it it thinks faster than humans, it’s extremely good at binary exploitation compared to humans. Well, I say ‘thinks’. It it kind of does stuff, whatever it does. I saw I saw your hesitation.
Linda (09:11)
Thank you
Josh (09:17)
It it predicts the next token in a way that mimics thinking faster than humans really think. Acceptable?
Linda (09:25)
Yep. Yep. Yep. It processes faster.
Josh (09:29)
Yeah. And it can it can exercise more hypotheses in a shorter time. which is kind of excellent for a large search space problem where it needs to go through all of these pieces of code, it needs to try all of these things, it needs to get the result and then act on that and iterate, iterate, iterate. It can do that faster than a human. because it can I mean one of the limits is it can just like read faster than a human. Like we have to go and read and understand something. It doesn’t, there’s no understanding step for the AI, right? Like once it’s on the chip, it can start predicting the next token. We have to like process it, understand it, sort of come to grips with the implications of what we’re seeing because we’re really thinking and understanding, whereas all it has to do is load that into the memory and then predict the next set of tokens.
Linda (10:26)
So it’s it’s
Josh (10:27)
So that’s sort of why we’ve seen things like this go so fast.
Linda (10:29)
It’s really just trying stuff repeatedly and continuously changing what it’s trying until something works and it knows when something has worked because the the the success criteria, the the goal is very well defined.
Josh (10:46)
Yeah, and there’s there’s a lot of this in its training data and it’s stuff that’s extremely hard for humans to understand. So unless you’re like really deep into computer science, binary exploitation is not it’s a very niche subject. It’s really difficult.
Lots of people in the industry can’t do it effectively, right? So the bar to be somewhat successful is lower than in some other industries where you know maybe a ton of it’s something humans are really good at. We kind of need machines to help with binary exploitation these days. Like people run things called fuzzers so they just exercise every code path in a piece of code and they say like well if I take this branch then this branch then this branch through the binary did it crash and then you record your crashes and you can work back from hypotheses. So Google does this on an unfathomable scale with important pieces of software that keep the internet online. There’s there’s a few projects like this. I think it’s called AlphaFuzz, if I’m not wrong. but what we’ve seen is I I I’m pretty sure this PHP2shell bug was a use after free vulnerability.
Linda (12:10)
What does that mean?
Josh (12:12)
so the, the technical details here might be wrong, because I’ve only had a few days to get up to date on this and I’m not really a binary exploitation guy. But
Linda (12:22)
We don’t really want the technical details anyway. For the purposes of the podcast, like a high level what does that mean from the point of view of the the people running the systems or like w what does it look like?
Josh (12:32)
Mm-hmm.
So rather than something classic like a buffer overflow where we’re writing memory directly to somewhere it shouldn’t be and then it’s getting executed, in a use after free, we are holding a pointer to some piece of memory, and then we are freeing that memory, giving it back to the system to be able to reallocate, and then we’ve, Wewve freed it, we’ve released it. It anything can be there now, but then we use that pointer again in some other piece of code.
Linda (13:09)
That’s cute.
Josh (13:09)
So it’s been used after it’s been freed. Yeah.
Linda (13:11)
Right.
Josh (13:12)
So by the time it’s used, we can no longer guarantee what’s going to be there.
Linda (13:18)
Yeah.
Josh (13:18)
And in the case of a use after free vulnerability, the attacker has put something there that they control. So that’s the code that’s going to be executed.
Linda (13:26)
Wait. that’s clever.
Josh (13:29)
And this is extremely hard to fuzz for, which is why the the robots keep finding so many. Like they’re they’re they’re getting really good at finding this bug class because it’s
Linda (13:42)
Hm.
Josh (13:43)
it takes sort of some interpretation. It it’s sort of like a state management thing, right? Like you’re you’re maintaining state from different parts of code, you’re not just seeing a crash, which was very easy to you know, well, not very easy. The people who write fuzzes are geniuses, but
Linda (13:59)
Ha ha ha.
Josh (14:00)
it was something we could do. Whereas a use after free kind of requires more of a manual audit. which was like at the scale it needs to be done for the amount of code that exists in the world and how complex it is, is just not until sort of now like practicable for like th the amount of intelligence that needs to be applied to it. You can you can kind of just chuck the robot at it and let it crunch through for hours. And it might find something of that shape.
Linda (14:32)
Wow. And because it doesn’t need
Josh (14:34)
Yeah.
Linda (14:34)
to be smart and and think about it, it can just keep throwing random shit until something sticks.
Josh (14:42)
Pretty much. With a use after free, it’s sort of detecting again, this is really not my area of expertise, but my understanding is it it’s sort of detecting the shape of what’s happening in the code.
Linda (14:57)
Right.
Josh (14:57)
And it’s it’s quite good at reasoning about that because there’s so much code in its training data and it’s easy to test.
Linda (15:02)
Well, yeah, it’s pattern matching.
Josh (15:05)
Yeah. So it it’s basically doing pattern matching at a massive scale and
Linda (15:09)
Mm.
Josh (15:11)
Funnily enough, the pattern matching pattern generating machine is quite good at pattern matching. So it’s
Linda (15:15)
Ha ha ha
Josh (15:17)
it’s done a good job of finding these vulnerabilities. which is
Linda (15:22)
Mm.
Josh (15:22)
why we’ve seen that timeline shrink by so much, which has been you know i it’s been dramatic in this case.
Linda (15:30)
Yeah. Yeah. There’s there’s a lot of noise about it. A lot of a lot of screaming, I suspect a lot of sleepless nights.
Josh (15:37)
And I I do want to mention that it has actually like it hasn’t just made things easier, right? Like for in my role as sort of simulating an attacker, right? I’m simulating the bad guys, is what my job tends to be. It’s made this step of the process easier. I don’t have to go
Linda (15:57)
Mm.
Josh (15:58)
and spend, you know, three days or a week reverse engineering this or anything like that, but when it got leaked, we saw scores of fake AI-generated reports on what had happened. And so
Linda (16:18)
wow.
Josh (16:19)
all of us are watching private GitHub channels and private Discord channels where we we like to talk about stuff like this, and we’re seeing, you know, this blog post, this blog post, this blog post, and they’re just wrong. They’re they’re
Linda (16:33)
Mm.
Josh (16:34)
hallucinations or they’re feasible claims that are just incorrect when tested. But because it’s the pattern matching robot,
Linda (16:44)
It’s very good at being plausible.
Josh (16:46)
yeah, it’s very, very good at being plausible. So we even posted one internally at work.
Linda (16:52)
Ha ha.
Josh (16:52)
And we were like, well, we have no idea if that’s right. And when I test it, it doesn’t work. But it looks very plausible. So we’ll just wait and see. And it’s just complete BS. It was just made up.
So it it’s become easier and harder, right? Because the defenders have to go and write detections for these vulnerabilities for their firewalls and stuff like that. And if they’re getting a bunch of fake stuff coming out, they have to weed through all of that until a credible source produces something. And the credible sources tend to move more slowly because they have to
Linda (17:29)
Yeah.
Josh (17:30)
maintain their credibility. Like
Linda (17:32)
And be careful and check things.
Josh (17:35)
Yeah, and they have to actually be right. Whereas, you know, Joe Schmoe on Twitter can just say something and get a bunch of attention and be completely wrong, but they you know, they got their attention.
Linda (17:48)
Yeah.
Josh (17:49)
Which is what we saw a lot. So there was a lot of distraction happening.
Linda (17:54)
It’s like there’s a prize for being first, but there’s no prize for being right.
Josh (17:58)
Yeah. It’s it’s very true. And it’s much easier to be first now and plausibly
Linda (18:04)
Mm.
Josh (18:06)
close in the in the realm of correct.
Linda (18:09)
Yep.
Josh (18:11)
So
Linda (18:11)
Right enough to look real, but wrong enough to be dangerous.
Josh (18:17)
Right enough for someone to skim the article and go, yeah, I’ll pass that on. And it’s just not true.
Linda (18:24)
Mm.
Josh (18:25)
So it’s been it it’s very interesting. It’s very it’s very different to how it used to be. Something like this would be private for a very, very long time before.
Linda (18:36)
Mm.
Josh (18:39)
This one has been I think fully disclosed now. It was I think it was found
It was found through AI originally. Like humans were involved. You have, you know, there’s people using AI, and then there’s people using AI, right? Like if you know the subject matter extremely well, you’re gonna produce much better outcomes. And so
Linda (18:59)
Yeah. Yeah.
Josh (19:01)
And so there were security experts, highly, you know, highly talented people, highly skilled people going through and using AI to look for vulnerabilities like this. And I think I saw the original discoverer saying I I was looking for this shape of vulnerability in PHP in these places with AI. That’s very,
Linda (19:23)
Right.
Josh (19:24)
very, very different to saying to the AI, go find me a huge security issue. Right.
Linda (19:31)
Yeah.
Yeah. Very targeted, very specific and as you said before, very clear goals, very obvious right answer. Like when you when you when you’ve got the right answer, you can tell, you can test it.
Josh (19:47)
Yeah. Like it either works or it doesn’t. So
Linda (19:50)
Mm.
Josh (19:52)
they say in the industry they say POC or GTFO, which is like publish a proof of concept script. So POC proof of concept or GTFO, which I’m sure everyone knows. So because it either works or it doesn’t. So
Linda (20:01)
Hm. Yep. Yeah.
Josh (20:07)
you either publish a POC and it works or stop talking about it, right? And
Linda (20:10)
Yeah. Yeah.
Josh (20:13)
the AI can write a POC. So it
Linda (20:17)
Freaky.
Josh (20:18)
it’s it’s very interesting in that regard.
Linda (20:21)
Hmm. That’s super interesting. Is there from the point of view of a of a cybersecurity expert, is there one thing that you wish everybody knew about data that would kind of change things for you or make life better if if everybody understood this one thing?
Josh (20:40)
from the point of view of a cybersecurity expert, I think people should be aware of the the sheer amount of data that is being collected on you all the time. it it’s like absolutely ridiculous the amount of data that’s being collected on you, by people who may or may not have asked permission. and they’re using it.
It sounds conspiratorial when you say it, but the the amount of tracking that can be done against a person from what would generally be considered like inconspicuous data is like ridiculous. Like there’s when you’re browsing different websites on your computer, the different encryption standards that your computer happens to support are fingerprinted. The way your mouse moves around the screen is you know, directly tied to your body’s morphology and your your brain to hand connection. So we can fingerprint how your
Linda (21:41)
Wow.
Josh (21:42)
mouse is moving on the page and then we can, you know, backtrack and have some data as to who you might be. You know?
Linda (21:50)
So never mind facial recognition, you’ve got mouse use recognition. That’s crazy.
Josh (21:55)
Yeah, so there’s even video game anti-cheat systems being developed. So when someone’s playing a competitive shooting game, they shoot in a particular way, they move their mouse in a particular way. when they get banned for cheating and then make a new account, we can fingerprint that mouse movement and say, Well, that’s the same person. We’re gonna ban them again.
Linda (22:16)
that’s wild.
Josh (22:18)
Yeah. So the the amount of data that’s being generated and what it can be used for is ridiculous.
And the the amount of data that’s sent when you go and visit something as simple as a website, like let let alone something like Facebook or Google who are putting cookies on every single page. So it’s not just their websites you’re visiting, it’s every website you’re visiting is getting extra signals. They can fingerprint you across everything you’re doing, across multiple devices. You know, they know which devices are in your house. They they they know they know where you live, they know what you buy, they know when you buy it, they know what made you decide to buy it better than you do.
So people should be considering that and I think gen generally for the purpose of protecting yourself in that regard, the best thing most people can do is like like sort of raising their voice about it because there’s not a whole lot you can do to actually protect yourself. Like you can use various privacy focused browsers and things like that to help. you can there’s some stuff you can do, right? Like you can go on and on and on about things that you can do to help protect your privacy, but it’s not really gonna stop until it becomes a huge issue for the companies who are doing it.
Linda (23:51)
So are you suggesting that we shout for regulation or just that we shame the companies into behaving better? Or both?
Josh (23:58)
I I mean I think I I don’t know if shame would work, right? I think
Linda (24:03)
They don’t have much.
Josh (24:05)
No, like I don’t think that’s gonna stop them. I I think there like there are some good things like the you know the right to be forgotten in various jurisdictions where you can go and say, I want you to delete everything you know about me. there there’s some good stuff like that, but if if more I feel like if most people knew the amount of data that was being collected on them, they would not give consent to that data being collected. So I I think people need to know how much data is being collected on them and what it’s used for and how it’s being sold. Like it’s extremely valuable to sell this data to advertisers or to insurance companies or anyone.
Beyond that, I would say like the way a lot companies, I do think make a genuine effort to a certain extent to protect people’s data because we have strong regulations that say you have to. if they didn’t, I don’t think they would protect it beyond their bottom dollar. But I think we I mean some companies, there are obviously some companies are better than others by a long way. But, you whenever you hand that data over to someone else, you’re you’re entrusting that personal information to another party with typically no visibility into how it’s being stored, how long it’s being kept for, what where it’s being kept, who has access to it on a daily basis, is it being sold to anyone else? you you have no idea. And as far as they’re concerned, you don’t really have a right to know.
Which I think’s kind of crazy.
Linda (26:01)
Yeah. Yeah, that’s wild. I I probably am more aware of that than the average person on the street and I had no idea about some of the stuff you’ve just described. That’s really disturbing.
Josh (26:19)
It’s it’s crazy.
Yeah, it it’s like your your computer, whatever you’re using is pretty much giving off a unique fingerprint every time you
Linda (26:30)
Mm.
Josh (26:31)
visit a page, whether you you know no matter what. so you you can be tracked pretty easily. so I I really think that’s probably not something we should be letting happen. Like if our government was
Linda (26:49)
Mm.
Josh (26:49)
doing that to us, we would have a huge problem with it. Even,
Linda (26:52)
Yeah.
Josh (26:53)
you know, there are there are problems with governments, but there’s at least some oversight and some accountability to the public in most like ideally. But with private companies that’s not even like the the idea to be accountable to to the public.
Linda (27:11)
Yeah.
Josh (27:12)
So I I think it would be yeah, I think more people should know how much is being tracked about them
Linda (27:20)
Yeah, know and care.
Josh (27:23)
Yeah, and care. Like I don’t think you can
Linda (27:25)
Speak up.
Josh (27:27)
Yeah, like people can be like, Well, I don’t care, like the companies know what I’m buying. But then if I if you talk about like dynamic price hikes on stuff that you need, all of a sudden you’re gonna care. And the the information that funds the dynamic or not funds but informs the dynamic price hike is the information that’s being collected on you now.
Linda (27:48)
Yeah. Yeah. And I was listening to Cory Doctorow talking about the treatment in the US of not just gig economy workers but also workers like nurses where companies will offer you know identify through credit checks the people in the most need of shifts and then offer them the least money and the worst shifts because they know they’re desperate enough to take it, which is a horrific human rights violation, but somehow legal.
Josh (28:29)
Yeah. Where it’s like the more information someone has over you, the more leverage they have over you. So just I I think it’s reasonable in a a world or a mindset where you respect sort of freedoms and like rights of human beings to try and reduce the amount of leverage that people have over one another.
Linda (28:55)
Yeah, that seems like a decent, you know, foundational platform that we seem to have let go.
Josh (29:04)
Yeah, it just moves so quickly. It’s really hard. And also now everyone’s typing everything about their entire lives into, you know, the robot. So
Linda (29:15)
Ha ha
Josh (29:16)
all of that data’s being kept as well, but people are volunteering that
Linda (29:19)
Yeah.
Yeah, that’s true. I I did an episode of Behind the News recently and one of the kids asked is my where does my data go when I type stuff into into a chatbot? And I said in my answer, you know, they they keep it and anyone at the company can look at it, probably other people as well, and you know, it might surface in answers to other people, probably not with your name attached, but you never know. And then the Character.AI thing came out where Character.AI went bust and all of their information, all of their data, all of their user conversations were suddenly up for sale and you have no recourse over that kind of no control over what happens to your data once it’s in the chatbot. Just horrific. Yeah.
Josh (30:08)
But you’re just handing it over. I’m seeing ads now for there are companies now offering to buy data dumps voluntarily handed over from companies of their internal like Slack or Teams channels for training purposes. And
Linda (30:26)
No
Josh (30:27)
they’re they’re offering hundreds of thousands of dollars to then sell these on to Frontier Labs for training.
Linda (30:37)
no that’s horrific.
Josh (30:39)
And there’s, I mean, it’s not unreasonable to think that a good number of people would take them up on their offer, right? Like
Linda (30:49)
Yeah.
Josh (30:49)
yet a a cash injection of hundreds of thousands of dollars to a lot of people and a lot of businesses would be you know, extremely hard to say no to.
Linda (31:02)
Boy. Just gets worse and worse, doesn’t it? I I I’m part of me is regretting this conversation. Like I was I was happier
Josh (31:10)
Well
Linda (31:10)
not knowing.
Josh (31:13)
I I am optimistic as well. I think in some ways we move in the right directions. I think. things like the right to be forgotten are great. I think people are getting but for example, pentest, we are the busiest we’ve ever been. w it’s everyone now, you know, has realized their stuff can be hacked. Like we
We’ve in sort of the security industry, we’ve been saying for years. the the expression we go back to is like obscurity is not security. So
Linda (31:51)
Yeah.
Josh (31:52)
just because someone doesn’t know something doesn’t mean it’s safe. And one of the other things AI is extremely good at is reverse engineering stuff. So the the sort of previous obscurity could even be like, well, my code is wrapped up into this executable binary, and that’s sort of a a lossy process in one direction. but we can we the the code still has to run, so it still has to do the same thing. And reverse engineering is just going back the other direction, back towards code that humans can read and AI is really good at that. So if you have some sort of executable or program or process that you think is secure because the way you distribute it means other people can’t look at it. Well they they can. and that was always the case before AI. It was insecure before the AI came out. It’s just no one had time to look. So you got away with it. Now people don’t even need
Linda (33:00)
Yeah. Yeah. Mostly.
Josh (33:03)
time to look. They just need, you know, the money to pay an AI to look and
Linda (33:09)
Mm.
Josh (33:10)
the outcomes can be profitable. So under our current system, if the outcome’s profitable for you to spend that money and have it look, then
Linda (33:17)
Mm.
Josh (33:18)
especially for criminals, like
Linda (33:20)
Mm.
Josh (33:21)
crime is extremely profitable, right? That’s why that’s why people do it. So
Linda (33:29)
Well, I’m glad you’re a white hat hacker, Josh. This is really disturbing. What are the worst data mistakes that you’ve seen? I mean it sounds like we’ve just talked about some of them in some ways.
Josh (33:45)
Some of them I I think I I like obviously I wasn’t in the professional world at the time of sort of our tech adolescence. So like maybe like the nineties and two thousands or something like that, where you know, sort of pre iPhone, like no one had a a computer on them twenty four seven, right?
Linda (34:08)
Yeah.
Josh (34:09)
Whereas now it it it’s almost impossible to imagine a world where you don’t have at least one computer you know in contact with your body 24-7.
Linda (34:17)
Mm-hmm, mm-hmm.
Josh (34:19)
and so I’ve sort of come into the world where that’s already true. People have at least one computer with them all the time. You know your car’s full of computers, your fridge has computers in it, you know, your watch is a computer, everything. So a lot of the mistakes we were making back then have largely gone away, which is great. So we’re seeing fewer and fewer cases of people just leaving a database completely exposed to the internet. But it still happens. It it’s still not not something you can write out of the equation, right? Like if you went and just started looking now, there are there are various platforms, there’s a thing called Shodan, for example, you can use this tool and you can say, all right, well, Microsoft SQL database runs on this port on TCP. Let me just go scan the internet for all of those and find them all. And then
Linda (35:21)
Mm.
Josh (35:23)
Shodan will help you with that. You could then spin up, you know, some Python script or whatever and say, I want to try admin password against all of those. And you’d you’d probably get
Linda (35:34)
Ha ha
Josh (35:35)
A couple things back and you could start, you know, there that is a still a problem. It probably the information you’d find is uninteresting
Linda (35:46)
Mm.
Josh (35:46)
for the most part, right? But you could still do it. and there are still there there’s modern equivalents of all of these things where maybe it’s not on the internet, but it’s in, you know, a special zone of Cloudflare’s infrastructure and the developer was doing a good job when they made it and they thought, well, this shouldn’t be on the internet, but all of our code runs in Cloudflare infrastructure, so it needs to be accessible from Cloudflare and their servers.
Linda (36:18)
Mm.
Josh (36:19)
But they fail to make the next leap, which is but just anyone on the internet can start a server inside Cloudflare and thereby access my database. So there’s
Linda (36:31)
Mm-hmm.
Josh (36:33)
things have been sort of, there’s another step. Like things are getting further apart. everyone knows not to use default passwords. It doesn’t mean they don’t.
Linda (36:43)
Yeah.
Josh (36:45)
Even if you’re not using a default password, a lot of software comes with a default account or like an admin account or a backup like disaster recovery account that people aren’t disabling. And then you can go
Linda (36:57)
Mm.
Josh (36:58)
use that data, and you only really know that if you go and read the manual or whatever that you’re deploying. a lot of these companies set up all this tech a very long time ago.
Linda (37:11)
Yeah.
Josh (37:12)
and it’s extremely expensive and time consuming and difficult to change that. Like once you have a bank running, you don’t really want to be like, well can we just turn it off for a couple days while we while we rejig it, right? You you can’t you you can’t turn a hospital off for a week.
Linda (37:27)
Yeah. So we can check it properly. Yeah. Yeah.
Josh (37:34)
to go do an infrastructure project. It it’s very it there’s not a simple way of doing it. So there are still like legacy ideas and systems around. and then like yeah obviously in my job like hacking into systems we we we get access to lots of data, right? But usually I like to think what we do is a bit more complex than just accessing something live. But that’s not to say it doesn’t still happen.
Linda (38:10)
Oof. It’s a wild west out there. That’s good, that’s good.
Josh (38:13)
It’s not as bad as it used to be. It’s it’s getting better every year. I I do genuinely believe every year that goes by, it’s getting better and better and better. I don’t want to be on here like fear-mongering.
Linda (38:26)
Yeah.
Josh (38:28)
It’s the best it’s ever been, and next year it’ll be better than that. So we’re on the up and up.
Linda (38:33)
That’s great, I like that. We’re getting we’re making progress. Have you ever seen data deliberately misused?
Josh (38:43)
deliberately misused only on like the news and the TV. I I
Linda (38:50)
Yeah. In the media.
Josh (38:53)
saw one on on Reddit, like there’s some really dreadful stuff on there. where I I forget what it is. It might be the infographics subreddit where it which
Linda (39:10)
huh.
Josh (39:10)
is kind of an interesting idea already. It’s like let’s take something interesting enough to grab people’s attention then summarize it down into like one plot, right?
Linda (39:20)
Mm.
Josh (39:20)
Which can be done in really interesting ways and can be lied about very easily. Right? And so I I was looking at one and I I can’t remember the exact statistics, but I looked at it and read the title across the top. It’s something about men versus women. And so I’m like okay, well, I I have some apprehensions about this chart already. so
Linda (39:49)
Brace yourself. Yep.
Josh (39:52)
I I look down in the bottom right and it’s made by like something along I I don’t want to accidentally misquote a real organization here, but it was something like familyfirstamericans.org or something like that. I was like, well that sounds a bit dog whistly. and so put that into Google and it goes, yeah, like hyper conservative alt-right think tank. But it’s sitting there at, you know, top of this month on the Reddit infographics page. People are scrolling past it, consuming it, consuming the exact message that it’s attempting to send to the person and then moving on. So they they have the that seed of an idea has now been implanted in their head. And that’s all that matters. It doesn’t matter if it was true, it doesn’t matter how the data was collected, where it came from, if the sample size was significant, if the experiment was conducted in a way where you can even draw the conclusions you’re claiming to be able to draw. None of this
Linda (40:51)
Yeah.
Josh (40:52)
is tested when you scroll past something on some sort of, you know, short form content or like somewhere like as much of a cesspool as Reddit is. That you you can’t but it doesn’t matter because it still leaves that impression on people.
Linda (41:09)
I I I think that’s a really fundamental point, the idea that, you know, we’re not asking those questions. We’re just when we scroll past something, we just take in the the message without, you know, you can’t ask critical questions about everything you see in a day, particularly everything you scroll past. We all scroll past so much that these messages get in even when we think they don’t.
Even when we we consider ourselves sceptical and and, you know, critical.
Josh (41:40)
Even then, if it’s something you even remotely resonate with, then it it just goes as a little like tick that affirms what I believe and move on. Like I’m now more deeply rooted in my established belief. Whether the belief is based in reality or not, like it doesn’t really matter. Like maybe your belief is sound and just and know, backed by science. But it doesn’t matter because you’ve just consumed something that’s affirmed your belief without any reason to do that. I I did we talked about it the other day, but I loved the one that came out the other day where the it was comparing all the different AI models, sort of like this, and
Linda (42:24)
yeah.
Josh (42:26)
the the vertical axis was model number, so like chat
Linda (42:30)
Yeah.
Josh (42:31)
GPT six is obviously higher than Claude 4.
Right. I thought
Linda (42:36)
Yeah.
Josh (42:37)
that was brilliant. That was such a good use of such a good way to take the piss out of the metrics that have been coming out.
Linda (42:44)
Yes, it was a beautiful piece of satire. And and the what got me was that I could well believe that that was something that a an AI company would put out, you know,
Josh (42:53)
Hundred percent.
Linda (42:54)
genuinely. It w I wasn’t entirely convinced initially that it was satire. So that’s the you know, you could believe that they would put something like that out and count.
Josh (43:01)
That’s the best form of satire, right?
Linda (43:03)
Yeah, it looks real.
Josh (43:06)
I when I saw it, I thought it the first time I saw it, I just scrolled past it and I was like, okay, some new like model benchmark. Cause I I I’m
Linda (43:13)
Yeah, yeah.
Josh (43:15)
I’m I’m hesitant to say this, but I am into AI. I find it interesting and it’s very,
Linda (43:21)
Yeah.
Josh (43:22)
very important to the work I do, so I stay up to date.
Linda (43:25)
Yeah.
Josh (43:27)
and so I was like, a new benchmark. I see benchmarks every day. I don’t believe three quarters of them. So I just moved past it. And then I looked at it the next time and saw the model number axis.
Linda (43:32)
Yeah.
Josh (43:36)
And it’s it’s so good.
Linda (43:40)
Very clever. what is the first question you ask then when you see a graph in the media or on Reddit as it may be? What what what do you what do you look for? What do you consider?
Josh (43:53)
It it’s actually it’s things that you taught me personally. but it’s
Linda (43:58)
Ha ha ha.
Josh (44:00)
the the three or four things, right? It’s like what are the labels on the axes?
Linda (44:06)
Yeah.
Josh (44:07)
and do they relate to the question being posed? where you know where do the axes start and what are the tick marks and scales? obviously if you’re getting a 1% year-on-year improvement, but you show that last year you were 60% and this year you’re 61%, and your graph starts at 55 and goes to 70, that’s gonna look like a massive jump when actually it’s more or less insignificant. and if you have like a logarithmic graph, obviously it changes changes the whole way you’re interpreting the relationship, right? and then number of data points, which is extremely hard for the like AI benchmarks. There’s a lot of charts coming out for those, and so many of them fall victim to like every bad principle that we’ve sort of ever discussed.
Linda (45:08)
Yeah.
Josh (45:09)
like sample size one like on a non-deterministic tool, like
Linda (45:13)
Yep. Yep.
Josh (45:15)
the the AI is non-deterministic. It’s gonna run differently every time you use it. So
Linda (45:20)
Yes.
Josh (45:21)
you can’t do one sample. It’s ridiculous to claim that you can. And then, like I said, with the other thing, with the think tanks, I like to check the source. and the source can’t be like source, just trust me. It has to be cited, it has to have come from somewhere. if someone’s not willing to put their name on it, then they didn’t do a good job, or at least I’m not willing to believe they did a good job. And then that organization has to have a strong reputation for you know, at the very least, attempting to tell the truth. Like, if if they’re well known to be pushing any kind of a agenda, that’s gonna be something you have to factor into your judgment. You you don’t have to discount the entire thing because of that. But
Linda (46:12)
Yeah. Yeah.
Josh (46:14)
you know, maybe if the if they’re well known for pushing a strong agenda in some direction, you should check their methodology and
Linda (46:20)
Mm-hmm.
Josh (46:21)
there there are there are different ways to ask a question right like you could say
Linda (46:24)
Yeah, definitely.
Josh (46:26)
and then you can interpret those answers in different ways that the it data science is real and the the method of like conducting studies has been developed in a specific way there are ethical guidelines on how to design surveys for example a lot of the like pop media data science stories are like we surveyed a gazillion people and this is what they think of this person. And it’s like, well your survey would not have passed any academic review. So there’s a reason we
Linda (47:01)
Yeah. Yep.
Josh (47:03)
have that academic review.
Linda (47:05)
My personal favorite is the the leading question style of survey,
Josh (47:10)
Mm.
Linda (47:11)
not how did you, you know, how did you rate the workshop on on a scale of, you know, terrible to amazing, but how amazing was the workshop? Or you know, how much
Josh (47:23)
yeah.
Linda (47:23)
do you hate this person or how much do you love this person gets a very different answer to how do you feel about this person, and kind of priming
Josh (47:31)
Yeah.
Linda (47:32)
for a particular response.
Josh (47:35)
Yeah, I mean and they do it all the time. And like e everyone does it all the time because
Linda (47:43)
Yeah.
Josh (47:45)
I mean it’s a broader problem in science and academia of when you set out to investigate a hypothesis you are hoping to be correct. And so
Linda (47:55)
Yes.
Josh (47:57)
one of the reasons we have all of the reviews and all of these things that are put in place, sort of the big machine around, you know, human knowledge and the the testing that goes into it is because we as people have an ego, and we would like to be correct, and we would like to be recognized for the hard work we did because we put
Linda (48:19)
Mm.
Josh (48:20)
years of our life into this hypothesis we don’t want to be wrong. And so even subconsciously, things are gonna slip in. you know, it giving everyone the benefit of the doubt, things will still
Linda (48:34)
Yeah.
Josh (48:35)
you’re going to read your data through a favorable lens, whether you want to or not, unless
Linda (48:42)
Yeah.
Josh (48:42)
you’re diligently applying the sort of academic rigour that you need to. I’m sure all of your
Linda (48:47)
A hundred percent.
Josh (48:48)
listeners I expect know this.
Linda (48:51)
And that’s one of the reasons why I get so intense about my projects being, you know, a built in part of the project is you critically evaluate your own work. You look for the flaws that you know are there. You don’t look to see is it right or wrong. You go, where are the flaws? Because there are flaws in there because it’s real work and there’s no such thing as, you know, a perfect solution to a real problem. And
And I think we would be a lot better off if we did if we had a lot more of that in education. So that the default is there are issues with my work because I am human and the data is complex and messy and the situation is real and you know, therefore there it’s it’s not there’s no such thing as a perfect answer. We’re gonna actually look for the flaws instead of going, Well, obviously it’s perfect, it’s mine and I’m amazing, you know, and just kind of moving on, which is
Josh (49:42)
It’s
Linda (49:43)
all too often the standard
Josh (49:46)
It is like I I don’t know actually off the top of my head if MMDL is tied into ADSEI or not. it is? Okay. But like one of the reasons Okay. Yes, okay.
Linda (49:57)
Yeah, it’s a product of the Australian Data Science Education Institute.
Josh (50:03)
but it’s like it I do think we would be in such a better situation at all levels. So like from within someone’s local, someone’s community, not like where they are physically, but their community of people they network with.
Linda (50:19)
Mm. Mm.
Josh (50:21)
not network in a LinkedIn way, but network in a human way. but at all levels
Linda (50:25)
Nice distinction.
Josh (50:27)
from sort of your your local social network to your local government to the country to the world, if people understood that statistics can be used to convincingly lie to you very, very easily, it is unimaginably easy and, b ack to the AI thing, it is really, really easy now to give some information to AI and say, I want you to prove that this data says this and it will do some lying and some shenanigans to make that happen for you because it is designed to be a yes man, right?
Linda (51:04)
Yeah.
Josh (51:05)
So now someone someone with no understanding of how to engineer data to lie to you can now take that data and use it very convincingly to lie to you because it’s
Linda (51:17)
Mm.
Josh (51:18)
the the the robot is so good at making things sound plausible.
Linda (51:23)
Yep. Yep. That’s that’s the bottom line, isn’t it? You’ve really pinpointed some of the key issues here. It’s been such a good conversation. And we are up to my favourite question, the last question, which is what excites you about data?
Josh (51:40)
I think there is like a huge amount of good that can be done for the world with the data we are collecting. there are challenges in aligning what is good for the world with what people are spending their time doing because people need to eat and feel secure before they’re willing to do things like that.
There are some really good companies coming out who are attempting to poach talent from, you know, people who might otherwise go to a weapons manufacturer or a a high frequency trading firm or something like that. and I I don’t necessarily think someone is like a a moral failure for doing that because at the end of the day they still need to make it through life, which is a difficult thing to do. But there
Linda (52:43)
Mm.
Josh (52:44)
are some really interesting companies being developed whose entire goal is to grab those people who would presumably rather be working on something that contributes positively to the world on a massive scale, and put them in extremely interesting, high-paying jobs so that they can get all of the outcomes they want from their life while also, you know, I I I think that’s kind of one of the issues is that the people who can do this are like if you’re an extremely good data scientist, there’s probably a lot of money to be made in mining or oil or weaponry, warfare, things that a lot of people casinos thing things that a lot of people would probably rather not be doing, but the the benefits make it so easy to and and you’re very abstracted from what you’re doing. Like there’s a there’s a lot of things happening here. It’s not as simple as like, well, you do XYZ, therefore you’re evil. but if we can get those people into positions where they’re getting those same benefits, they feel the same level of security and comfort.
But they’re also doing good things for the world. I think data can take us an extremely long way with society. Like, I I I do think like a lot of utopian ideas are entirely possible with what we already have. there are enough resources on Earth right now to go around. Right? we just we have a logistics problem. And we have
Linda (54:26)
Mm-hmm.
Josh (54:28)
Like a an allocation problem and these are problems where if we put the data in front of some very, very smart people, that I’m very confident can be solved. We we just have to get those smart people in front of that data with the right tools and i it I do think things will be fine.
Linda (54:48)
Such a great place to end. Thank you so much, Josh. This has been a wonderful conversation. I I I knew it was gonna be good but this has way exceeded my expectations.
Josh (54:58)
No, thank you. it was it was fun. I was very nervous but it was it was a lot of fun.
Linda (55:04)
I’m glad. I’m glad you enjoyed it.
Outro (55:07)
Thanks for listening to Make Me Data Literate. You can find more episodes at ADSEI.org/podcast and you can support our work at givenow.com.au/ADSEI. Have a great day.
