The following essay is reprinted with permission from The Conversation, an online publication covering the latest research. The last few months have witnessed the unraveling of the remarkable life sciences company Theranos, culminating in the latest news that federal regulators may ban founder Elizabeth Holmes from the blood-testing industry for at least two years. The company is also facing a federal criminal investigation into whether it misled investors about its technology and company operations. How has this widely acclaimed biomedical innovator fallen so far, so fast? Theranos’ revolutionary claim that won over investors was that it could accurately run tests using a small amount of blood taken from a poke in the patient’s finger, instead of a syringe full from a needle stuck in a vein. The idea was that dozens of tests, such as cholesterol and thyroid hormone levels, could be run on a single, tiny blood sample. Theranos has had medical laboratory experts—including me—scratching their heads for some time. Having worked quite a bit in development of innovative medical tests, I knew what the company was promising just didn’t make a lot of sense. Theranos’ much-hyped blood-testing technology Theranos wowed investors, journalists and even groups like the Cleveland Clinic and Walgreens with what you might call an “iMedicine” vision of blood testing. Basically, the idea was this. A few drops of blood from a fingertip are collected into a “nanotainer” collection tube and analyzed on the company’s proprietary machine, named after famed inventor Thomas Edison. How exactly the Edison devices work is unknown. But the claim was that many—possibly dozens—of tests could be run on those few drops of blood. From a clinical perspective, this was always concerning, as such a shotgun approach to medical testing is actually very bad medicine. It is well-known that running numerous tests without symptoms or signs of disease invariably leads to false positive results. Questions about the company’s technology came to a head in October. The Wall Street Journal alleged that Theranos might actually be performing the majority of its tests using traditional machines, the kind already in use in labs across the country, instead of its own much touted Edison devices. And then the FDA called the nanotainer an “uncleared medical device.” The company stopped using its signature collection tube, except in the single test that had been cleared by the FDA. And, by the way, a machine capable of measuring many molecules on a drop of blood might seem like a breakthrough, unless you are someone familiar enough with life sciences technology to know that such capability has been invented numerous times. For instance, in my laboratory, we have a small handheld analyzer called an I-STAT. It can do 25 different tests on very small amounts of blood. It was developed more than 20 years ago and is commonplace today. Blood from a finger isn’t same as blood from a vein Let’s leave the technical issues aside for a moment and focus on those drops of blood from a fingertip. The vast majority of FDA-approved medically important laboratory tests are based on blood taken from a vein, not from the finger. In fact, the package inserts for medical tests cleared by the FDA say something like “blood should be collected using standard venous blood collection techniques.” To understand why that matters, let’s start with some simple physiology. The human body can be considered a series of compartments; the concentration of any given molecule in blood or tissue fluid may vary from one compartment to the next. A small molecule, such as glucose, can move easily between these compartments, and any bodily fluid can generally be used to test its concentration. This is why testing blood sugar with a finger prick works. But large, medically important molecules like proteins and lipids are not always found in uniform concentrations throughout the body. The composition of blood from finger pricks from the same person can vary, a problem that doesn’t happen in blood taken from a vein. When you lance a fingertip, you get both blood and tissue fluid, and this means that the concentration of molecules may be different than if the blood sample comes from a vein. The point is that blood taken from a finger would be considered another bodily fluid by the FDA, and any tests using finger prick blood would need to cleared by the FDA. According to news reports, at one time Theranos was seeking FDA clearances for as many as 120 tests. Even if its technology actually works, hundreds of FDA clearances would have required hundreds of clinical trials, a process that would have taken years to complete. Theranos has gotten only one test cleared by the FDA. That test—for Herpes infection—is for the detection of antibodies, not a measurement of their concentration. Tests for the presence or absence of a molecule are much simpler than those that quantify its concentration. In March, researchers at the Mount Sinai School of Medicine published a secret shopper study comparing Theranos to Quest and LabCorp, two major medical testing companies. Overall, more of the results from Theranos were outside the normal range, indicating a need for further medical testing. This is a worrisome result because the secret shoppers were healthy adults. But it’s also difficult to interpret because we don’t know if Theranos used its Edison machines or ran diluted samples on conventional analyzers. Millions invested without any proof? Why didn’t investors and journalists dig more deeply, such as by demanding a head-to-head comparison of Theranos’ Edison machine to standard chemistry analyzers? Part of the problem seems to have been the secrecy surrounding these types of startups. Theranos always asserted that it had to operate in “stealth mode” to protect its lead in breakthrough technology, which means that there was literally no peer-reviewed information out there about its technology. But the leading explanation seems to be that they were enthralled by the company’s charismatic young founder. Laudatory magazine cover articles about Holmes have been so numerous that the phenomenon has itself been a topic of discussion. Theranos’ board of directors also lacked anyone with expertise in laboratory testing or medical diagnostics. This should have been a warning sign. Show me the data Just a few weeks before regulators proposed banning Holmes and Theranos President Sunny Balwani from the blood-testing industry, the company tried to remedy this by bulking up its medical advisor board with well-qualified experts in chemistry, pathology and clinical chemistry. It’s hard to imagine these experts would have signed on amid all the bad publicity and allegations without demanding proof that the technology works, but who knows? It still remains possible that Theranos has discovered a breakthrough technology that can do hundreds of lab tests on a drop of fluid from a patient’s finger. But even if this increasingly unlikely prospect is a reality, Holmes’ erstwhile acolytes need to remember the lessons learned from the pantheon of past pied pipers and summed up by statistician W. Edwards Deming:
In God we trust; all others must bring data.
Norman A. Paradis does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond the academic appointment above. This article was originally published on The Conversation. Read the original article.
The following essay is reprinted with permission from The Conversation, an online publication covering the latest research.
The last few months have witnessed the unraveling of the remarkable life sciences company Theranos, culminating in the latest news that federal regulators may ban founder Elizabeth Holmes from the blood-testing industry for at least two years. The company is also facing a federal criminal investigation into whether it misled investors about its technology and company operations.
How has this widely acclaimed biomedical innovator fallen so far, so fast?
Theranos’ revolutionary claim that won over investors was that it could accurately run tests using a small amount of blood taken from a poke in the patient’s finger, instead of a syringe full from a needle stuck in a vein. The idea was that dozens of tests, such as cholesterol and thyroid hormone levels, could be run on a single, tiny blood sample.
Theranos has had medical laboratory experts—including me—scratching their heads for some time. Having worked quite a bit in development of innovative medical tests, I knew what the company was promising just didn’t make a lot of sense.
Theranos’ much-hyped blood-testing technology
Theranos wowed investors, journalists and even groups like the Cleveland Clinic and Walgreens with what you might call an “iMedicine” vision of blood testing.
Basically, the idea was this. A few drops of blood from a fingertip are collected into a “nanotainer” collection tube and analyzed on the company’s proprietary machine, named after famed inventor Thomas Edison. How exactly the Edison devices work is unknown. But the claim was that many—possibly dozens—of tests could be run on those few drops of blood.
From a clinical perspective, this was always concerning, as such a shotgun approach to medical testing is actually very bad medicine. It is well-known that running numerous tests without symptoms or signs of disease invariably leads to false positive results.
Questions about the company’s technology came to a head in October. The Wall Street Journal alleged that Theranos might actually be performing the majority of its tests using traditional machines, the kind already in use in labs across the country, instead of its own much touted Edison devices. And then the FDA called the nanotainer an “uncleared medical device.” The company stopped using its signature collection tube, except in the single test that had been cleared by the FDA.
And, by the way, a machine capable of measuring many molecules on a drop of blood might seem like a breakthrough, unless you are someone familiar enough with life sciences technology to know that such capability has been invented numerous times. For instance, in my laboratory, we have a small handheld analyzer called an I-STAT. It can do 25 different tests on very small amounts of blood. It was developed more than 20 years ago and is commonplace today.
Blood from a finger isn’t same as blood from a vein
Let’s leave the technical issues aside for a moment and focus on those drops of blood from a fingertip.
The vast majority of FDA-approved medically important laboratory tests are based on blood taken from a vein, not from the finger. In fact, the package inserts for medical tests cleared by the FDA say something like “blood should be collected using standard venous blood collection techniques.”
To understand why that matters, let’s start with some simple physiology. The human body can be considered a series of compartments; the concentration of any given molecule in blood or tissue fluid may vary from one compartment to the next.
A small molecule, such as glucose, can move easily between these compartments, and any bodily fluid can generally be used to test its concentration. This is why testing blood sugar with a finger prick works.
But large, medically important molecules like proteins and lipids are not always found in uniform concentrations throughout the body. The composition of blood from finger pricks from the same person can vary, a problem that doesn’t happen in blood taken from a vein.
When you lance a fingertip, you get both blood and tissue fluid, and this means that the concentration of molecules may be different than if the blood sample comes from a vein.
The point is that blood taken from a finger would be considered another bodily fluid by the FDA, and any tests using finger prick blood would need to cleared by the FDA. According to news reports, at one time Theranos was seeking FDA clearances for as many as 120 tests. Even if its technology actually works, hundreds of FDA clearances would have required hundreds of clinical trials, a process that would have taken years to complete.
Theranos has gotten only one test cleared by the FDA. That test—for Herpes infection—is for the detection of antibodies, not a measurement of their concentration. Tests for the presence or absence of a molecule are much simpler than those that quantify its concentration.
In March, researchers at the Mount Sinai School of Medicine published a secret shopper study comparing Theranos to Quest and LabCorp, two major medical testing companies. Overall, more of the results from Theranos were outside the normal range, indicating a need for further medical testing. This is a worrisome result because the secret shoppers were healthy adults. But it’s also difficult to interpret because we don’t know if Theranos used its Edison machines or ran diluted samples on conventional analyzers.
Millions invested without any proof?
Why didn’t investors and journalists dig more deeply, such as by demanding a head-to-head comparison of Theranos’ Edison machine to standard chemistry analyzers?
Part of the problem seems to have been the secrecy surrounding these types of startups. Theranos always asserted that it had to operate in “stealth mode” to protect its lead in breakthrough technology, which means that there was literally no peer-reviewed information out there about its technology.
But the leading explanation seems to be that they were enthralled by the company’s charismatic young founder. Laudatory magazine cover articles about Holmes have been so numerous that the phenomenon has itself been a topic of discussion.
Theranos’ board of directors also lacked anyone with expertise in laboratory testing or medical diagnostics. This should have been a warning sign.
Show me the data
Just a few weeks before regulators proposed banning Holmes and Theranos President Sunny Balwani from the blood-testing industry, the company tried to remedy this by bulking up its medical advisor board with well-qualified experts in chemistry, pathology and clinical chemistry.
It’s hard to imagine these experts would have signed on amid all the bad publicity and allegations without demanding proof that the technology works, but who knows?
It still remains possible that Theranos has discovered a breakthrough technology that can do hundreds of lab tests on a drop of fluid from a patient’s finger. But even if this increasingly unlikely prospect is a reality, Holmes’ erstwhile acolytes need to remember the lessons learned from the pantheon of past pied pipers and summed up by statistician W. Edwards Deming:
Norman A. Paradis does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond the academic appointment above.
In God we trust; all others must bring data.
This article was originally published on The Conversation. Read the original article.
