COVID-19 serology testing will be available soon. The tests have two important uses. The first is to check if someone has been exposed to the virus. Although not yet proven, most experts believe that these people will develop some immunity, lasting months or years. This is the idea behind the immunity certificates’ some countries are thinking about. Because of uncertainty about immunity, and problems with test performance, this is not being proposed here.
The second important use is to assess how many people in the community have been exposed. It is likely that all or most recovered people will have made antibodies. If we knew the proportion of the whole population who had antibodies, this would help us know how many people had no symptoms and were not detected. It would also help plan our path out of our work and social restrictions.
The TGA has given emergency approval to a number of simple point-of-care (POC) tests. Some might be suitable for the first purpose (“Who has had it?”), but not likely for the second (population screening). This is because of two aspects of test performance. Test sensitivity shows the proportion of affected people with a positive result. Test specificity shows the proportion of unaffected people with a negative result. A simple POC test might have 95% sensitivity and 95% specificity. So it would give a false negative result in 5% of affected people and a false positive result in 5% of non-affected people. Some tests might perform better, some worse. We don’t yet have full, independent, evaluation data on the TGA’s ‘emergency approved’ tests.
We can use Bayes’ theorem, devised by the 18th century British clergyman, Thomas Bayes, to help interpret the likelihood of results being true or false in any particular person. Consider a group of mine workers (or an Australian city) where, unknown to us, 5% have had COVID-19. This is the ‘prior’ or ‘pre-test probability’ we use in our calculation. Testing is done with a simple finger-prick blood sample at a mining site (or at a doctor’s surgery). Its sensitivity and specificity, as explained above, are both 95%. Bayes theorem tells us that, in this situation, we can be 99.8% confident that a negative result is true and that most people will give negative results. However, half of the positive results will be false-positives. That test is not good enough.
Now consider state-of-the-art tests, under development and not yet available, run on sophisticated laboratory instruments. Their sensitivity is 100% and their specificity is high at 99.8%. We expect just one false positive result for every 500 people who have definitively never had the disease and who should be negative. Using this improved test, we could now have 100% confidence in a negative test result and 96.3% confidence in a positive test result in our mine workers. Were we issuing immunity certificates, we would probably accept this test.
Can we use this new test for the second purpose – testing the general population to see how many have been exposed to COVID-19? Let’s check the numbers. We’ve had 6,800 cases so far, say 7,000. If we assume that only one-third were detected, the real number would be closer to 20,000. Let’s assume that all those 20,000 people developed antibodies. As the Australian population is 25,000,000, the proportion (‘prevalence’) of people with antibodies would be: 20,000/25,000,000 = 0.08% – or 800 people in every 1,000,000. Assume our new test has a sensitivity of 100% and a specificity of 99.8%. If we tested a random sample of one million Australians, this is what we would see.
Let’s assume that somehow, we manage to test the 800 people who do have antibodies first. Because our test has 100% sensitivity, we will detect them all and get 800 true-positive results. Next, we test the remaining 999,200 people without antibodies. If our test were perfect, we would get all negative results. However, our test has only 99.8% specificity, so we can expect to get one false-positive for every 500 people tested. When we have finished testing the remaining 999,200 people, we will have mostly negative results, but 1,998 will have false-positive results. There would be more than twice as many false-positives than true-positives. As a result, we would get the wrong answer to our question: “How many people have been infected?” Our test would tell us that 800 + 1,998 = 2,798 people per million had been infected, whereas the true number would be 800 per million. We have overestimated the proportion of people with antibodies by more than three times.
The lesson is that we have to be careful how we use and interpret diagnostic tests. The Reverend Bayes could never have imagined that his theorem would one day, in the setting of a global pandemic, be used to help guide governments’ decisions.
Dr Bruce Campbell, BSc (Hons), MBBS (Hons), FRCPA, FAACB
Chief Editor of Lab Tests Online-AU