Is Fitbit unfit by default?

Profile picture for user gonzodaddy By Den Howlett May 23, 2016
Studies suggest that Fitbit heart rate trackers produce unreliable results. This is not the first time that Fitbit has come under scrutiny. We believe this topic needs a more objective assessment.
Fitbit Charge HR wearable on wrist by @philww
Fitbit Charge HR

At the beginning of the year, Fitbit was hit by a class action lawsuit claiming that the company's PurePulseTM Tracker technology doesn't work and leads to 'wildly inaccurate results.' The case is interesting to us for multiple reasons.

Fitbit, in common with many other device makers, seeks to limit users' rights to launch class action suits. The plaintiffs'  lawyers make the case that the way Fitbit attempts to do this is unlawful. Assuming they get over that hurdle, then what of the evidence?

The initial lawsuit provides details of how three people compared what their Fitbit heart rate (HR) trackers were saying with other methods of measuring heart rate and found considerable disparity in the results. The lawsuit says that the consistency of inconsistent (sic) results is reflected by 'scores' of other complainants, citing reviews on Amazon.

Yesterday, a study (PDF) undertaken by two PhD's with specialties in exercise physiology and human physiology and metabolism on behalf of the plaintiff's lawyers was released. As you might expect, the study supports the lawsuit. While the report is laden with academic jargon, the conclusions are unequivocal:

Overall, the results of this investigation demonstrate that the PurePulseTM technology integrated in Fitbit’s heart rate monitoring devices is not a valid method for heart rate measurement, and cannot be used to provide a meaningful estimate of a user’s heart rate.

In layman's terms, the study claims the trackers are useless. The lawsuit claims that people who relied upon Fitbit's explicit advertising are denied the claimed benefits of HR tracking. For its part, Fitbit was quick to come back at reporters and say the study:

"lacks scientific rigor and is the product of flawed methodology ... and was conducted with a consumer-grade electrocardiogram – not a true clinical device, as implied by the plaintiffs’ lawyers." The statemetn also contend "there is no evidence the device used in the purported 'study' was tested for accuracy."

Fitbit "stand[s] behind our heart-rate monitoring technology and all our products, and continue to believe the plaintiffs’ allegations do not have any merit. We are vigorously defending against these claims, and will resist any attempts by the plaintiffs’ lawyers to leverage a settlement with misleading tactics and false claims of scientific evidence.”

I'll come back to the science bit in a moment because it is important to understanding the way Fitbit picks and chooses the facts it presents and then spins them away from science.

In August 2014, I recorded a grungy/fuzzy video (shown at the end of this story) where I proved that by simply swinging my arms, a Fitbit Force will provide a false reading for steps taken. As might be expected, trolls decided that while the video might be compelling, I was cheating and so anyone with any sense could forget what I was claiming. Of course I was cheating. I deliberately foreshortened events for the purpose of video to demonstrate something that I had observed while working at my desk over a period of hours. Regardless. the point remains the same. It is relatively easy to show that the Fitbit Force produces inaccurate and/or misleading results.

In my case it doesn't matter that much. I stopped using Fitbit devices once a handful of them stopped working. But the widespread use of these devices raises important questions about the perceptual benefits of health related technology.

Does accuracy matter?

I have heard and read many similar arguments about these types of device. The arguments broadly state that the devices are not meant to act as health monitors in the clinical sense but are meant to act as indicators of whatever they are measuring. To my simple mind, that's a bit like a manufacturer saying the speedometer in a car isn't meant to tell me the speed at which I am traveling but only provides an approximation. I wonder how well that would go down if I was served with a speeding ticket, or worse still, had an accident that caused injury?

Fitbit - and many others - are attempting to provide devices that encourage healthier practices and to that extent should be praised for bringing to market devices that customers want to use. In that sense, they have been very successful. However, the disconnect between healthy lifestyles as a medical imperative and devices which appear on their face to provide false information strikes me as dangerous for several reasons.

While the lawsuit is commercial in its leanings, reports point to some of the consequences that could arise out of the study:

This study will scare the many athletes - serious competitors and weekend warriors alike – who aim to train at certain heart rates. If the devices over-report heart rates, users will have trained at lower heart rates than they wanted to achieve. If the devices under-report, users may be straining to reach heart rates beyond their optimal peak levels. Which can end badly.

These observations are not unreasonable and at some point I'd like to see results from polls that ask the fundamental question: "Do you consider your fitness tracker to provide an accurate measure of [fill in the blanks here?]" Clearly the class action plaintiffs expected so, otherwise why would they be concerned at apparent inaccuracies?

Aligning to health issues

In the latest earnings call, James Park, CEO Fitbit said:

Fitbit devices are changing lives and in some cases saving lives to the power of data, inspiration and guidance. Recently Fast Company Magazine highlighted the amazing transformation of an individual in one of our corporate wellness program. Brett Broviak participated in the Indiana University Health program, shared with Fast Company how he was borderline diabetic with high cholesterol and blood sugar levels when he joined the wellness program in 2014. He quickly became a star of the program walking 1 million steps in a single month which is 30,000 steps a day eventually losing 30 pounds and lowering his cholesterol to normal levels after also changing his diet and becoming more active.

Just last month a story was published in the medical journal Analyst of Emergency Medicine reviewing the case of a 42 year old man who presented to the emergency room with newly diagnosed atria fibrillation of unknown duration. The patients showed up at the ER after grand mal seizure confused and unaware that his heart was in trouble when trouble began. Doctors noticed he was wearing a FitBit and at the patient’s permission used heart rate data from the patient’s Fitbit app, they quickly determined when the arrhythmia started and help decide the best course of treatment needed to get his heat beat under control.

While I would like to emphasize that Fitbit’s current products are not medical devices and do not themselves diagnose or treat medical conditions, we hear those stories regularly.

In short, Fitbit's own reporting to the markets is overtly aligning device results with positive medical outcomes but with a huge caveat at the end. That's not tenable when weighed against user expectations and results under discussion unless you are prepared to suspend disbelief. But there is more. Park continued:

With healthcare cost continuing to rise of employers, insurers and consumers, no surprise that innovation and employee wellness programs continues to be increasingly important for both the private and public sector alike. Fitbit is helping to lead that front. In fact last month VP & GM at Fitbit Wellness, Amy McDonough was invited to testify in front of Congress the House Education and the Workforce Subcommittee and Health Employment Labor and Pension around innovations in employee sponsored healthcare.

Your health costs in their hands?

Reading between the lines, you can easily surmise that at some point, if it is not already happening, Fitbit will sell the data it collects for all manner of health related purposes, with an obvious target of the healthcare insurers as the first port of call. But if that data is inaccurate then what does that mean for insurance premiums? What does that mean for those that don't use fitness trackers? What does that mean for companies that wish to establish wellness programs and policies? To my mind, the entire field is fraught with risk, the cost of which will almost certainly fall upon the consumer.

It is broadly accepted by those I speak with that the US healthcare system is shambolic. Do we need another twist in the tail coming from a marketing led commercial enterprise that sells a product of questionable value? To my mind that's marginally short of insane but it is an entirely possible outcome.

My critics will likely say - "Aaah Howlett, WTF do you know?" And they'd be right. And that's the point. Nobody knows for sure although I would argue that we should seek answers about the accuracy of these devices along with discovering what's happening with the data collected. Those are legitimate lines of inquiry.

And, without wishing to sound overly dramatic, it is worth bearing in mind that one of the most vaunted health related technology 'advances' was recently exposed as a fraud.