- Research
- Open access
- Published:
Field evaluations of four SARS-CoV-2 rapid antigen tests during SARS-CoV-2 Delta variant wave in South Africa
Diagnostic and Prognostic Research volume 7, Article number: 14 (2023)
Abstract
Background
Rapid antigen tests detecting SARS-CoV-2 were shown to be a useful tool in managing the COVID-19 pandemic. Here, we report on the results of a prospective diagnostic accuracy study of four SARS-CoV-2 rapid antigen tests in a South African setting.
Methods
Rapid antigen test evaluations were performed through drive-through testing centres in Durban, South Africa, from July to December 2021. Two evaluation studies were performed: nasal Panbio COVID-19 Ag Rapid Test Device (Abbott) was evaluated in parallel with the nasopharyngeal Espline SARS-CoV-2 Ag test (Fujirebio), followed by the evaluation of nasal RightSign COVID-19 Antigen Rapid test Cassette (Hangzhou Biotest Biotech) in parallel with the nasopharyngeal STANDARD Q COVID-19 Ag test (SD Biosensor). The Abbott RealTime SARS-CoV-2 assay was used as a reference test.
Results
Evaluation of Panbio and Espline Ag tests was performed on 494 samples (31% positivity), while the evaluation of Standard Q and RightTest Ag tests was performed on 539 samples (13.17% positivity). The overall sensitivity for all four tests ranged between 60 and 72% with excellent specificity values (> 98%). Sensitivity increased to > 80% in all tests in samples with cycle number value < 20. All four tests performed best in samples from patients presenting within the first week of symptom onset.
Conclusions
All four evaluated tests detected a majority of the cases within the first week of symptom onset with high viral load.
Introduction
Diagnostic testing has proven to be imperative for the management of SARS-CoV-2/COVID-19 in the context of reducing transmission and outbreak control [1]. While the gold standard RT-PCR test is highly sensitive and specific, there are several disadvantages including cost, complexity and length of the process, and need for specialised equipment and trained personal. The use and availability of rapid antigen tests (RDTs) for the diagnosis of SARS-CoV-2 infection have significantly increased over the last year of the pandemic. RDTs are affordable, fast (10–30 min), simple, and do not require specialised laboratory facilities or highly trained personal. Although their sensitivity is lower compared to a laboratory-based RT-PCR, antigen-based RDTs can detect infection early following symptom onset when the viral load is high, thereby offering quick screening and detection of SARS-CoV-2/COVID-19 among high-risk groups [2].
As of May 2022, South African Health Products Regulatory Authority (SAHPRA) has approved 53 RDT test for use in South Africa [3]. While the World Health Organization (WHO) recommends a minimum of 80% sensitivity and 97% specificity for rapid antigen diagnostics tests to be approved, there is significant variation in RDT performance depending on the study settings [4,5,6,7,8,9,10] As an example, the reported performance of commonly used Standard Q COVID-19 Ag test varies between 28.7 [11] and 89.2% [12] depending on the prevalence and patient group. Furthermore, there continues to be a limited number of reports on RDT field performance in low- and middle-income country (LMIC) settings. Here, we evaluate the performance of four rapid antigen tests in comparison with the Abbott RT-PCR assay during the B.1.617.2 Delta variant wave in KwaZulu-Natal, South Africa.
Methods
Study participant recruitment
Rapid antigen test evaluations were performed between July and December 2021 through drive-through testing centres in Durban, South Africa. Adult participants (age = > 18) meeting any of the following criteria were enrolled in the study: tested COVID-19 positive in the previous 7 days, the presence of COVID-19 symptoms in the previous seven days, exposed to COVID-19 5–10 days ago, healthcare worker, or doctor referral for testing. Drive-through testing centres were freely accessible with no referral necessary for testing. Data on screened out individuals that did not fit the study enrolment criteria is not available. Two separate evaluation studies were performed: first, the Espline SARS-CoV-2 Ag test [Fujirebio, nasopharyngeal (NP)] [13] was evaluated in parallel with the Panbio COVID-19 Ag Rapid Test Device (Abbott, nasal) [14]; this was followed by the evaluation of RightSign COVID-19 Antigen Rapid test Cassette (Hangzhou Biotest Biotech, nasal) [15] in parallel with the STANDARD Q COVID-19 Ag test (SD Biosensor, NP) [16] on a different group of participants. The Abbott RealTime SARS-CoV-2 assay (target sequences in the SARS-CoV-2 RdRp and N genes of the SARS-CoV-2 genome) [17] was used as a reference test. The Abbott RealTime SARS-CoV-2-positive results are reported with cycle number (CN) values that are equivalent to cycle threshold values more commonly used by other assays [18]. Following informed consent, participants filled in a questionnaire on basic demographic and clinical data and provided samples for the study. Study participants were provided with the results of the South African Health Products Regulatory Authority (SAHPRA) and approved rapid antigen test on site, and Abbott RT-PCR results were reported within 24 h of sample collection. At the time of the study, the nasal Panbio Ag test and NP STANDARD Q Ag test were SAHPRA approved. The study protocol was written before recruitment began. The protocol was not published. Ethics for the parent study was obtained on 24 March 2020. The evaluated tests were selected and provided by the Foundation for Innovative New Diagnostics (FIND) based on the availability and evaluation needs at the time of the study. Minimum of 50 positive cases for each test was required for evaluation. Test evaluations were continued until study ran out of available kits.
Sample collection and processing
Study participants provided three swabs: one nasal and two nasopharyngeal. Nasal swab was collected first in order to avoid cross-contamination between sites, followed by a NP swab for second rapid antigen test, and followed by the NP swab for SARS-CoV-2 RT-PCR. Swabs were collected, and rapid antigen tests were performed and interpreted by trained medical staff on site. The results of the tests were interpreted and recorded by two staff members independently. Swabs for RT-PCR were sent to the central laboratory at room temperature without additives. Swab was resuspended in 2 ml of viral transport media (VTM) and processed within 3 h of sample collection. All assays were performed as per manufacturer’s protocol.
Statistical analysis
Statistical analysis was performed using SPSS version 27 and GraphPad Prism version 8.3.1 (GraphPad software, La Jolla, CA, USA). Study analysis was prespecified by FIND. Test performance characteristics [sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV)] were calculated in reference to Abbott RealTime SARS-CoV-2 assay results. Wilson’s score method was used to calculate the 95% confidence intervals to assess the level of uncertainty induced by sample size. Test performance was assessed across different categories including the presence and duration of symptoms (no symptoms, symptoms = > 7 days, and symptoms < 7 days) and CN values (< 20 and < 25) as indicator of viral load and infectiousness [19]. In case of missing data, a complete-case analysis approach was used. The measure of agreement between the assays was evaluated using Cohen’s kappa coefficient [20]. A t-test was used to assess differences in CN values between true-positive and false-negative results.
Results
Study sample characteristics
The evaluation of nasal Panbio and NP Espline Ag tests was done on 494 samples (Table 1) between 21st of July and 19th of August 2021. The median age of study participants was 34 [interquartile range (IQR) 24–47] with 57.29% of participants being female. The SARS-CoV-2 positivity was 31.00% with median CN value of 9.06 (IQR 5.90–16.90), with all positive samples having a CN < 31. Majority of the study participants (57.36%) presented for testing during the first week post-symptom onset.
The evaluation of NP Standard Q and nasal RightSign Ag tests was done on 539 samples (Table 1) collected between 15th of September and 8th of December 2021. The median age of study participants was 36 (IQR 24–50) with 51.76% being female. The SARS-CoV-2 positivity was 13.17% with a median CN value of 15.08 (IQR 11.53–23.86), with all positive samples having a CN value < 31. Majority of the study participants presented for testing during first week post-symptom onset (64.01%).
Test performance evaluation
The overall performance of nasal Panbio and NP Espline Ag tests is summarised in Sup. Table 1 and Fig. 1. The overall sensitivity and specificity of nasal Panbio Ag test were 67.97% (95% CI 60.22–74.85) and 98.53% (95% CI 96.61–99.37), respectively. The overall performance of NP Espline Ag tests was slightly higher with overall sensitivity and specificity of 72.00% (95% CI 64.33–78.67) and 99.71% (95% CI 98.35–99.95), respectively. The sensitivity of both tests increased in samples with lower CN values (increased viral load): for samples with CN < 20 sensitivity of nasal Panbio Ag, test was 80.16% (95% CI 72.35–86.18), and sensitivity of NP Espline Ag test was 82.11% (95% CI 74.40–87.88). Both tests performed best in samples from individuals presenting during the first week of symptom onset, with sensitivities of 74.16% (95% CI 64.20–82.12) for Panbio and 76.14% (95% CI 66.26–83.83) for Espline Ag tests, and worst in individuals presenting more than 7-day post-symptom onset, with sensitivities of 37.50% (95% CI 21.16–57.29) for Panbio and 47.37% (95% CI 27.33–68.29) for Espline Ag test.
The performance of NP Standard Q and nasal RightSign Ag test is summarised in Sup. Table 2 and Fig. 1. The overall sensitivity and specificity of NP Standard Q-test were 60.56% (95% CI 48.94–71.11) and 99.79% (95% CI 98.80–99.96), respectively. The overall sensitivity of nasal RightSign Ag test was 63.38% (95% CI 51.76–73.63) with specificity of 100.00% (95% CI 99.19–100.00). The sensitivity of both tests increased in samples with lower CN values, being 87.23% (95% CI 74.83–94.02) for both Standard Q and RightSign Ag test in samples with CN < 20.
The kappa coefficient ranged between 0.72 and 0.78 indicating substantial agreement between the tested RDTs and the Abbott RT-PCR assay. As expected, majority of false-negative (FN) results for all 4 tests occurred in samples with higher CN values (Sup. Figure 1). There were 4 invalid test results with Espline Ag test (0.8%), with no invalid results for the remaining three tests.
Discussion
In this study, we determined the sensitivity and specificity of four RDTs for the detection of SARS-CoV-2 virus in respiratory specimens during the Delta wave of infections in Durban, South Africa.
The sensitivity of the four evaluated tests ranged from 60.55 to 87.23% with high specificity (ranging from 83.33 to 100%). The observed sensitivity of the evaluated tests is comparable with previously published studies from different settings [2, 21]. As reported previously, we observed similar results between the nasal and nasopharyngeal assays performed on equivalent samples [22, 23]. The sensitivity of each of the RDTs increased in samples with lower CN values, increasing above 80% in samples with CN < 20. As previously reported, all four tests performed best in samples from individuals presenting within the first week of symptom onset when the SARS-CoV-2 viral load is highest [24,25,26,27]. While the rapid antigen tests have often been criticised for low sensitivity and high rate of false-negative results, the majority of the false-negative results are observed in samples with higher C threshold/number values (low viral load) that likely have a limited potential for fulling further viral transmission [2, 24, 28, 29]. In fact, antigen-based RDTs were shown to correlate better with replication-competent SARS-CoV-2 compared to RT-PCR [30] further supporting the use of rapid antigen tests in identifying individuals who are at high potential to transmit SARS-CoV-2. Additionally, low cost and scalability represent an important advantage over standard RT-PCR tests, especially in low- and middle-income countries.
There are several limitations to our study. We did not have access to participant vaccination status and the presence and timing of previous natural infections, and we therefore could not assess the impact of previous immunity on SARS-CoV-2 viral load and rapid antigen test performance. Even though viral sequencing was not available in our study, the period overlaps with the Delta wave of infections in KwaZulu-Natal [31]. Additionally, we have no data on patients that were screened out due to not meeting the study enrollment criteria. While none of the evaluated tests satisfied the WHO requirements for the > 80% sensitivity in the overall sample group, they are still a valuable tool in identifying infected individuals within the first week of symptom onset and those with high viral loads and could play an important role in limiting transmission and controlling the COVID-19 pandemic. Rapid antigen tests remain a useful tool for rapid screening for COVID-19 in congregate settings as well as for “test to work” strategies in order to reduce/slow down spread of the virus. This study provides valuable information of the performance of rapid antigen tests in drive-through centres in South Africa.
Availability of data and materials
De-identified patient-level data can be accessed by contacting the corresponding author with a detailed description of the research question.
References
Baxter C, Abdool Karim Q, Abdool Karim SS. Identifying SARS-CoV-2 infections in South Africa: balancing public health imperatives with saving lives. Biochem Biophys Res Commun. 2021;538:221–5.
Dinnes J, Deeks JJ, Berhane S, Taylor M, Adriano A, Davenport C, et al. Cochrane COVID-19 Diagnostic Test Accuracy Group. Rapid, point‐of‐care antigen and molecular‐based tests for diagnosis of SARS‐CoV‐2 infection. Cochrane Database Syst Rev. 2021;3(3):CD013705. https://doi.org/10.1002/14651858.CD013705.pub2.
SAHPRA. Medical devices and in vitro diagnostics test kits. 2022; Available from: https://www.sahpra.org.za/medical-devices/medical-devices-and-in-vitro-diagnostics-test-kits/. [Cited 30 Oct 2022].
World Health Organization. Advice on the use of point-of-care immunodiagnostic tests for COVID-19: scientific brief, 8 April 2020. World Health Organization; 2020. https://apps.who.int/iris/handle/10665/331713.
Krüger LJ, Gaeddert M, Köppel L, Brümmer LE, Gottschalk C, Miranda IB, et al. Evaluation of the accuracy, ease of use and limit of detection of novel, rapid, antigen-detecting point-of-care diagnostics for SARS-CoV-2. medRxiv. 2020.10.01.20203836; https://doi.org/10.1101/2020.10.01.20203836.
Wang Y-H, et al. Evaluation of the diagnostic accuracy of COVID-19 antigen tests: a systematic review and meta-analysis. J Chin Med Assoc. 2021;84(11):1028–37.
Bello-Chavolla OY, et al. Diagnostic performance and clinical implications of rapid SARS-CoV-2 antigen testing in Mexico using real-world nationwide COVID-19 registry data. PLoS One. 2021;16(8):e0256447.
Navero-Castillejos J, et al. Diagnostic performance of six rapid antigen tests for SARS-CoV-2. Microbiology Spectrum. 2022;10(2):e02351-e2421.
Prestedge J, Williamson DA. The performance of rapid antigen tests against SARS-CoV-2 variants. Lancet Infect Dis. 2023;28:S1473.
Konstantinus I, et al. Laboratory and field evaluation of the STANDARD Q and Panbio™ SARS-CoV-2 antigen rapid test in Namibia using nasopharyngeal samples. PLoS One. 2022;17(9):e0269329.
Lee J, et al. Clinical performance of the standard Q COVID-19 rapid antigen test and simulation of its real-world application in Korea. Ann Lab Med. 2021;41(6):588–92.
Kim HW, Park M, Lee JH. Clinical evaluation of the rapid STANDARD Q COVID-19 Ag test for the screening of severe acute respiratory syndrome coronavirus 2. Ann Lab Med. 2022;42(1):100–4.
Fujirebio. Fujirebio Espline SARS-CoV-2 2020. Available from: https://www.lumiere32.sg/p/fujirebio-japan-espline-sars-cov-2. [Cited 2020].
Abbott. Panbio COVID-19 Ag Rapid Test Device. 2020. Available from: https://www.globalpointofcare.abbott/en/product-details/panbio-covid-19-ag-antigen-test.html. Cited 2020.
Biotest. RightSign COVID-19 Antigen Rapid test Cassette (Hangzhou Biotest Biotech) 2020. Available from: http://en.biotests.com.cn/proDetail.aspx?id=258#product. Cited 2021.
Biosensor, S. STANDARD Q COVID-19 Ag test (SD Biosensor). 2020. Available from: https://www.sdbiosensor.com/product/product_view?product_no=241. Cited 2020.
Abbott. RealTime SARS-CoV-2: package insert (09N77–095). 2020; Available from: https://www.fda.gov/media/136258/download.
Hirschhorn JW, et al. Verification and validation of SARS-CoV-2 assay performance on the abbott m 2000 and alinity m systems. J Clin Microbiol. 2021;59(5):e03119-e3120.
Singanayagam A, et al. Duration of infectiousness and correlation with RT-PCR cycle threshold values in cases of COVID-19, England, January to May 2020. Eurosurveillance. 2020;25(32):2001483.
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.
Akingba OL, et al. Field performance evaluation of the Panbio rapid SARS-CoV-2 antigen assay in an epidemic driven by the B.1.351 variant in the Eastern Cape, South Africa. J ClinVirol Plus. 2021;1(1–2):100013.
Lindner AK, et al. Head-to-head comparison of SARS-CoV-2 antigen-detecting rapid test with self-collected nasal swab versus professional-collected nasopharyngeal swab. Eur Respir J. 2021;57(4):2003961.
Samsunder N, et al. Clinical evaluation of severe acute respiratory syndrome coronavirus 2 rapid antigen tests during the Omicron wave in South Africa. J Infect Dis. 2022;226(8):1412–7.
Cevik M, et al. SARS-CoV-2, SARS-CoV, and MERS-CoV viral load dynamics, duration of viral shedding, and infectiousness: a systematic review and meta-analysis. The lancet microbe. 2021;2(1):e13–22.
Wyllie AL, et al. Saliva or nasopharyngeal swab specimens for detection of SARS-CoV-2. N Engl J Med. 2020;383(13):1283–6.
Zou L, et al. SARS-CoV-2 viral load in upper respiratory specimens of infected patients. N Engl J Med. 2020;382(12):1177–9.
Killingley B, et al. Safety, tolerability and viral kinetics during SARS-CoV-2 human challenge in young adults. Nat Med. 2022;28(5):1031–41.
van Kampen JJ, et al. Shedding of infectious virus in hospitalized patients with coronavirus disease-2019 (COVID-19): duration and key determinants. MedRxiv. 2020.
Paul G, et al. Klinische Performance eines neuen SARS-CoV-2-antigen-tests in der Notaufnahme eines Maximalversorgers. 2020.
Pekosz A, et al. Antigen-based testing but not real-time polymerase chain reaction correlates with severe acute respiratory syndrome coronavirus 2 viral culture. Clin Infect Dis. 2021;73(9):e2861–6.
Tegally H, et al. Rapid replacement of the Beta variant by the Delta variant in South Africa. MedRxiv. 2021.
Acknowledgements
The authors would like to thank all the study participants for their participation and the District Health Services and frontline healthcare providers for their ongoing support throughout the study. A special thanks to the study staff for the field work and laboratory analysis and for their dedication to these studies.
Funding
This project was funded as part of FIND’s work as co-convener of the diagnostics pillar of the access to COVID-19 tools (ACT) accelerator, including support from the Swiss Agency for Development and Cooperation (grant number 81066910). Research presented here was partly funded by the European & Developing Countries Clinical Trials Partnership (EDCTP): RIA2020EF-2928 — mobilisation of research funds for COVID-19 as a Public Health Emergency and the South African Department of Science and Innovation (DSI) top-up funding to strengthen EDCTP COVID-19 regional networks.
Author information
Authors and Affiliations
Contributions
Designed the study, AS and NS; performed the experiments, AS, NS, SN2, TGM, SR, and SNM; analysed the data, AS, GL, NS, TGM, and LL; wrote the first draft of the paper, AS, NS, SN1, and GL; supervised clinical and/or experimental aspects of the study, AS, CC, KN, ABMK, NS, SAK, and QAK; and all authors contributed to the editing and finalisation of the manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
All study participants provided written informed consent. The KwaZulu-Natal Biomedical Research Ethics Committee approved the study (BREC approval No: BREC/00001195/2020 and BREC/00003106/2021). Research presented here was performed in accordance with Good Clinical Practice guidelines and the Declaration of Helsinki.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Additional file 1: Supplementary figures: Fig. 1.
Differences in SARS-CoV-2 CN values between true positive and false negative samples for A) Panbio COVID-19 Ag Rapid Test Device (nasal) B) Espline SARS-CoV-2 Ag test (NP) C) STANDARD Q COVID-19 Ag test (NP) D) RightSign COVID-19 Antigen Rapid test Cassette (nasal). Supplementary tables: Table 1. Estimations of test performance for Panbio COVID-19 Ag Rapid Test Device (nasal) and Espline SARS-CoV-2 Ag test (NP) with respect to RT-PCR CN values (A) and presence/duration of symptoms (B). *Represents overall sensitivity and specificity (CN values for all positive samples were <31). Table 2. Estimations of test performance for STANDARD Q COVID-19 Ag test (NP) and RightSign COVID-19 Antigen Rapid test Cassette (Nasal Swab) with respect to RT-PCR CN values (A) and presence/duration of symptoms (B). *Represents overall sensitivity and specificity (CN values for all positive samples were <31).
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Samsunder, N., Lustig, G., Ngubane, S. et al. Field evaluations of four SARS-CoV-2 rapid antigen tests during SARS-CoV-2 Delta variant wave in South Africa. Diagn Progn Res 7, 14 (2023). https://doi.org/10.1186/s41512-023-00151-3
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s41512-023-00151-3