No lab test is 100% accurate. Hopefully most have about 90-99% sensitivity and specificity. Doctors know that some results will be wrong, so we are always weighing all our evidence, and not relying entirely on a lab result to make a diagnosis.
We test sick patients. Health insurance does not pay for screening tests in healthy patients, with certain notable exceptions. The combination of symptoms and lab results allows us to be mostly right.
When you start to screen healthy patients, who don't have symptoms, and you are trying to make a diagnosis based exclusively on a lab test, there are numerous possible pitfalls. Below is an excellent discussion of how baked-in test limitations can lead you wildly astray.
Taken verbatim from Sebastian Rushworth, MD's blog:
"As mentioned, the sensitivity of the PCR test seems to be around 88% . A good value for the specificity is harder to determine, but it’s somewhere between 88% and 100%, so if we assume a specificity of 94% (halfway between the two values) we’re probably not far off.
Let’s say the disease is spreading rampantly through the population, and one in ten people are infected at the same time. If we test 1,000 people at random, that will mean 100 of those people actually have covid, while 900 don’t. Of the 100 who have covid, the test will successfully pick up 88. Of the 900 who don’t have covid, the test will correctly tell 846 that they don’t have it, but it will also tell 54 healthy people that they do have covid. So, in total 142 people out of 1,000 are told that they have covid. Of those 142 people, 62% actually have the disease, and 38% don’t.
That’s not great. Four in ten people getting a positive test result don’t actually have covid, even in a situation where the disease is so common that 10% of people being tested really do have the disease.
Unfortunately, it gets worse. let’s assume the disease is starting to wane, and now only one in a hundred people being tested actually has covid. If we test 1,000 people, that will mean ten will really have covid, while 990 won’t. Of the ten who have covid, nine will be correctly told that they have it. Of the 990 who don’t have it, 931 will be correctly told that they don’t have it, while 59 will be incorrectly told that they do have the disease. So, in total, 68 people will be told that they have covid. But only 9 out of 68 will actually have the disease. To put it another way, in a situation where only 1% of the population being tested has the disease, 87% of positive results will be false positives.
There is another thing about this that I think is worth paying attention to. When one in ten people being tested has the disease, you get 142 positive results per 1000 people tested. But when one in a hundred has the disease, you get 68 positive results. So, even though the actual prevalence of the disease has decreased by a factor of ten, the prevalence of PCR positive results has only decreased by half. So if you’re only looking at PCR results, and consider that to be an accurate reflection of how prevalent the disease is in the population, then you will be fooled, because the disease will seem to be much more prevalent than it is.
Let’s do one final thought experiment to illustrate this. Say the disease is now very rare, and only one in a thousand tested people actually has covid. If you test 1,000 people, you will get back 61 positive results. Of those, one will be a true positive, and 60 will be false positives. So, even though the prevalence of true disease has again decreased by a factor of ten, the number of positive results has only decreased slightly, from 68 to 61 (of which 60 are false positives!). So by looking just at positive PCR tests, you can easily be convinced that the disease is continuing to be roughly as prevalent in the population, even as it goes from being present in one in a hundred people to only being present in one in a thousand. The rarer the disease becomes in reality, the less likely you are to notice any difference in the number of tests returning positive results.
I want to restate this again, in a slightly different way, to make sure the message sinks in. As the disease drops enormously, by a factor of 100, from affecting one in ten to one in a thousand tested people, there is little more than a halving in PCR positive results, from 142 to 61. So a huge reduction in real infections only causes a small reduction in PCR confirmed “cases”. In fact, the disease could vanish from the face of the Earth, and you would still be getting 60 positive results for every 1,000 tests carried out!
The same trend is seen even if the PCR test were to have a much better specificity than we are estimating here, of say 99% . Here’s a quick illustration, since I don’t want to tire you with too many more numbers. If one in ten has the disease and you test 1,000 people, you will get back 97 positive results, of which 88 will be true positives and 9 will be false positives. If one in 100 has the disease, you will get back 19 positive results, of which 9 will be true positives and ten will be false positives. If one in 1,000 has the disease, you will get back 12 positive results, of which 11 will be false positives.
So, even if the test has a very high specificity of 99%, when the virus stops being present at pandemic levels in the population and starts to decrease to more endemic levels, you quickly get to a point where most positive results are false positives, and where the disease seems to be much more prevalent than it really is.
As you can see, the less prevalent the disease is in reality, the more likely the test is to generate a false positive result, and the less useful the test is as a method for figuring out who actually has covid. And the less prevalent the disease is, the more prevalent it will seem to be in relation to reality. If decisions about covid continue to be made largely based on what PCR tests show, we might never be able to call off the pandemic!
And that, ladies and gentlemen, is why PCR positive cases are a very poor indicator of how prevalent covid is in the population, and why we should instead be basing decisions on the rates of hospitalization, ICU admission, and death. If we just look at the PCR tests, we will continue to believe that the disease is widespread in the population indefinitely, even as it becomes less and less common in reality."
Now, I (Meryl Nass) must add that currently, real cases ARE going up in the US, based on hospitalizations, ICU admissions, deaths. I don't for a second deny that.
But high total case counts are partly due to the problems inherent in the test process. The US is testing 1.5 million people a day, and about 160,000 people a day are testing positive. How many thousands are false positives?
No comments:
Post a Comment