Biostatistical Modeling, Estimation, and Decision-making Support for COVID-19 (June 5)
Biostatistical modeling, estimation, and decision-making support have been playing an important role in responses to COVID-19 challenges. Lu Tang and Andriy Bandos will discuss statistical considerations involved in modeling the epidemic progression and in the use of COVID-19 related tests for estimation and decision making support.
Watch on YouTube or click below
Additional Q and A from the 6/5 seminar
Are there prediction models taking into account the variability of the number of COVID-19 tests that were conducted every day?
A recently updated version of the model by the Institute for Health Metrics and Evaluation (IHME) claimed to have adjusted for the variability in testing capacity and populations tested, and models testing as one of the factors related to disease transmission. However, data on everyday testing may still be highly variable (see for example number of tests reported by the CDC) and often requires smoothing techniques and other statistical approaches for improving estimates. One workaround is to use more elaborated modeling of the disease progression based only on the most reliable data, e.g., daily reported COVID-19 death data, which are believed to have better measurement quality than case and testing data. Combined with appropriately detailed models, these approaches could lead to rather accurate projections at least in the near future. A list of forecasts models and their brief descriptions can be found on the CDC forecasting website.
What is the error rate for testing COVID-19 now? Has the US adopted the group/pooled testing procedure?
It could be misleading to describe the error rate of COVID-19 testing in terms of one or two numbers. For example, RT-PCR virus tests receiving Emergency Use Authorizations (EUA) by FDA typically demonstrate almost perfect performance in laboratory experiments. Yet, in real-world testing, the false-negative rate of virus tests highly depends on the type and timing of the sample collection, decreasing from 100% at the beginning of the infection to 20% and lower several days after typical symptom onset. At the same time, the false positives of the virus tests are practically negligible. These performance characteristics allow interpreting the “positive” virus test result as very likely ongoing COVID-19 infection, while cautioning against interpreting the sole “negative” result as the absence of the infection (see also the response to question #4).
It is also important to note that there are many different variants of the same test as well as types of tests (i.e., virus, antibody or serology, and imaging tests). They all have somewhat different performance characteristics and influential factors.
The group/pooled testing has not been universally adopted in the United States or other countries, although occasional large scale use does occur (e.g., testing 6.5 million people in Wuhan, China). In the United States, some laboratories use pooled testing (e.g., Nebraska Public Health Laboratory), and CDC alludes to the possibility of pooling strategy in small groups. As more information on the characteristics of the available tests becomes available, more effective and wide-spread application of group testing is likely to occur.
Have the virus testing and antibody testing improved for COVID-19 as more and more data being collected on different possible symptoms and to influence the gold standard?
It can be expected that more recently developed tests are somewhat more accurate, as manufacturers often try to improve upon, or at least reproduce, the results of previously developed tests. (Comparisons to the previously authorized tests are often a part of an application for the EUA.) It is also true that the availability of additional data, including the results of other tests, helps alleviate imperfections of a “gold standard” (i.e., the reference standard used for evaluating performance characteristics of tests). The use of additional (other than investigated) tests for establishing a better reference standard is also sometimes employed in validation studies (EUA Serology Test Performance).
With time, as more data on real-world testing and validation studies become available, we are learning more about the performance characteristics of the available tests, which will help with better application of tests for possibly upcoming challenges.
Would you recommend multiple different testings for viral load to detect whether the person is COVID-19 positive?
Because of the non-negligible errors, multiple testing can help increase the sensitivity or/and specificity of the resulting diagnostic panel. For virus tests, additional testing is used to increase the sensitivity of identification of the current COVID-19 infection to generate more reliable “negative” results (e.g., CDC recommends using two tests spaced by at least 24 hours as a part of determining the resolution of infection). For serology tests (for past infection), additional tests for antibodies to a different viral protein can be used to improve the accuracy of either result (e.g., EUA Serology Test Performance). In general, results of multiple tests can be more elaborately combined to improve the overall or decision-specific accuracy of the resulting panel. These combinations rely heavily on the knowledge of individual and joint characteristics of the tests.
Viral load (quantity of the virus in a fixed sample) determines whether the virus test result is “positive”, via a so-called threshold cycle (Ct). Quantitative tests can indicate the level of viral load in addition to the qualitative “positive”/”negative” result. Higher loads are expected to be associated with higher infectiousness, and thereby can drive specific intervention decisions. Combining the results of several quantitative, as compared with qualitative, tests offers greater potential for tailoring the performance characteristics to support decision making.
Resources/hyperlinks in order of appearance above:
CONVERSATIONS ABOUT COVID-19 | PITT PUBLIC HEALTH SEMINAR SERIES
Each Friday this May and June, Pitt Public Health experts will have a conversation about different aspects of the COVID-19 pandemic and then address questions from the seminar attendees. The goal of the COVID-19 seminar series is to provide our school community, alumni, and the public with credible information about the virus and the associated health impacts.
6/13/2020