Some months ago I bought myself a smart watch. Recovering as I am from both long covid and hip surgery, I wanted to track my health and try to use the data to optimise my recovery. My sleep is also not great, so I was hoping to track and optimise that, too.
There are a lot of ways to tweak your health and fitness, and I love to apply data to the problem. Does it matter if I have a few drinks with dinner? Does it make a difference how much exercise I do during the day? How can I get fitter, stronger, healthier, and more energetic? A smartwatch can’t exactly answer those questions, of course, but I was hoping it could at least help me figure out what helps and hinders my progress.
The rummage I did through reviews and comparisons suggested that they were all much the same in terms of accuracy and features, so I went with a Garmin device that had a pretty analog face.
Checking my sleep score each morning, I have been horrified, but not entirely surprised, to find that my sleep varies from bad to awful, with an occasional outlier of “fair”. The watch gives a sleep score out of 100, and I was rather surprised that it didn’t seem to correlate as well as I’d hoped with how I felt on waking. Sometimes I felt like I’d slept well, but my score was bad, and sometimes the other way around – my watch declared that my sleep was fine, but I felt like a victim of advanced sleep deprivation torture. I noticed early on, too, that if I woke feeling ok but my sleep score was bad, I’d immediately feel more tired – my watch was brainwashing me into feeling worse!
The app connected to my watch gives me a breakdown of the factors it uses to derive my sleep score, including duration, stress, and how much deep, light, and REM sleep I got, as well as how much of the time I was awake or restless.
Being of a naturally sceptical – one might say heretical – turn of mind, I set out to find out the formula that takes those values and turns them into a score out of 100. Sadly, though, it seemsthat the formula is proprietary. It looks as though it places a lot of weight on the “stress” value, the calculation of which is, wait for it, also proprietary. The trouble with proprietary formulae is that they can’t be independently tested and verified. Does a high “stress” score on my watch actually mean I’m stressed? Who knows!
I’ve noticed that my own perception of the quality of my sleep seems to map better onto my watch’s estimate of the amount of deep and REM sleep I got, rather than the sleep score with its mysterious stress weighting, so I looked into that, too. Now this is where I start to get cranky. I found a study that compares smart watches with the best scientific approaches we have. Frankly this stuff has not been studied nearly enough, but here’s what that study found: While most smart watches are around 88-89% accurate at figuring out whether we are asleep or awake (frankly, that’s higher than I expected, but lower than I’d like – how hard can it be!?), they typically achieve only 50% accuracy at detecting the phase of sleep we are in.
Fifty percent. These watches report minute by minute data on the way we are sleeping where any given data point has a fifty fifty chance of being completely wrong. You might as well toss a coin!
So here we have a sleep score that some company has made up and carefully NOT scientifically tested, based on a stress score that some company has made up and carefully NOT scientifically tested, plus a formula that some company has made up and carefully NOT scientifically tested (I’m sensing a trend), together with sleep phase data that has a 50-50 chance of being accurate.
Excuse me, but what!?!? This stuff is fabricated, and then marketed to us as useful, relevant, accurate data. It’s almost as though the tech industry is not interested in our actual wellbeing AT ALL. Only in how much cash they can extract from us in pursuit of it.
The Garmin website says “Advanced sleep tracking in compatible Garmin devices takes into account multiple factors to help you understand your sleep. In addition to the basics, such as when you fell asleep and when you woke up, you can see times when you were awake and how much time you spent in key sleep stages (light, deep, REM). You will also see when those stages occurred during the night.”
No. You can’t see any of that. You can see some guesses at that, which have a 50-50 chance of being right. I’m using Garmin as my example here, because I have a Garmin device, but there’s little difference between companies and devices. Some are shinier and have more bells and whistles, but at best they are guessing the sleep values that they report as real data, and they are not being open and honest about it.
If you have a smart watch to track your sleep, would you have bought it if it had openly confessed in its marketing materials that it was, at best, around 50% accurate? How useful is it to be using these devices to track things that they can’t reliably track? What does that mean for our attempts to optimise our wellbeing? Is this made up data leading us to made up conclusions?
Once again it seems that we are letting the tech industry fool and beguile us with shiny toys. We want to believe that the shiny toys have the power to change our lives for the better. When it comes to health tracking, the Smart in Smartwatch appears to refer to the marketing, not the results.
