Personal Science Week - 231109 Variability

Blood test results with high variability

Quantitative answers to personal science questions are only as good as the methods used to collect them. Unless your tests accurately reflect the underlying phenomenon, you won’t learn anything useful.

This week we look at some disturbing facts about common lab tests.

In PS Week 231026 we noted that our results from an at-home urine test were significantly different in the evening than in the morning. We also reminded readers that the CLIA lab certification process, while consistent for a specific lab, doesn’t imply that results will be comparable across labs. But surely the common blood tests we get from our annual physical are reliable, right?

Long-time readers know that we’re big fans of the at-home blood testing company, SiPhox Health, which sells a $95/month subscription that gives you results from about 20 blood markers, including all of the common ones used in most physical exams. Plus you get insulin, sex hormones, and Vitamin D. LabCorp will charge upwards of $400 for all those tests, and you don’t even have to leave your house.

At the time of my review, I noted that the SiPhox results mostly matched what I got from a LabCorp test I took at the same time — with one notable exception: SiPhox said my Vitamin D was a healthy 45 ng/mL, but LabCorp said I was well under 30 ng/mL — considered dangerously low. If LabCorp is right, then I probably need to urgently increase my Vitamin D, but the SiPhox result says the opposite. Who’s right?

But then SiPhox CEO, Michael Dubrofsky, showed me a chart from the life sciences company BioRad. When a lab wants to create a new diagnostic, they’ll want to test it against a known control, such as the BioRad Liquichek, which is essentially a chemical compound guaranteed to be a pure form of whatever you are targeting. I was shocked to learn that these control chemicals, while consistent for a single lab, can show significantly variability across different labs.

Remember: a lab that has passed CLIA inspection is legally allowed to claim they are measuring something so long as they do it consistently. CLIA does not make claims about whether that measurement is actionable one way or another. Different labs can pass CLIA inspection even with dramatically different results on the same specimen.

In fact, that’s what happened here. When the control was compared against a broad set of identical blood serum, the results are highly variable. The same specimen yields a dangerously-low 10 on some labs, and a comfortably-health 30+ on others, with everything in between. Whose results can you trust?

Source: BioRad

This situation is more serious than a simple question of which result is “correct”. Remember that this high variability applies to scientific research labs too. If a large peer-reviewed study makes a health claim, you don’t know if that applies to you unless you know exactly which lab performed the study. In the case of Vitamin D, where the consensus seems to be that anything under 30 ng/mL is “low”, you can’t compare your own results unless you know precisely which lab did the analysis.

I don’t know how much of an issue this is for other types of tests. Lipids like cholesterol, I’m told, are far easier to measure than a complicated molecule like Vitamin D, so there is less leeway in how a lab reports its results.

Nevertheless, consider my skepticism raised even further the next time I’m tempted to change my behavior strictly on the basis of one test result.

It’s true for cholesterol too

This is as good a time as any to remind readers of a fantastic research article published in 2019 by Azure Grant and Gary Wolf with support from Quantified Labs. Twenty volunteers recruited from the Quantified Self community took multiple measurements of their blood cholesterol at designated intervals throughout a single day. The result:

Within-a-day variability resulted in 47% of individuals crossing at least one risk category (i.e., low risk to moderate risk, or moderate risk to high risk) in TC (Total Cholesterol) or HDL-c, while 74% of individuals crossed at least one risk category for triglycerides (n = 19 participants for all within-day measures). All individuals crossed at least one risk category in at least one output.

In other words, if you make judgements about your cholesterol based on a single measurement, it’s very possible that you are misjudging your risk category.

As with the Vitamin D example above, keep in mind that the vast majority of research studies — including virtually every one that you or your doctor is quoting — base their conclusions on participants who have been tested a single time under specific conditions.

The conclusion: be skeptical, ask questions. Don’t assume that whatever you read in the peer-reviewed literature applies to you.

About Personal Science

Professional scientists are busy doing everything that anyone must do when they have a job. They have bosses who don’t always understand or appreciate them, colleagues who are trying to outcompete them for promotions, and the many inevitable tradeoffs that come when you do something in order to make a living.

As a personal scientists you’ll have your own biases, but with one advantage over the professionals: Nobody cares more about your health or well-being than you and your loved ones. Use that to your advantage: perform your own experiments, ask questions, don’t be afraid to try new things.

As always, please let us know if you have other topics you’d like us to cover, or leave a comment.