A Large-Scale Clinical Validation Study Using nCapp Cloud Plus Terminal by Frontline Doctors for the Rapid Diagnosis of COVID-19 and COVID-19 pneumonia in China
Charles A Powell,
Posted 11 Aug 2020
medRxiv DOI: 10.1101/2020.08.07.20163402
Posted 11 Aug 2020
Background The outbreak of coronavirus disease 2019 (COVID-19) has become a global pandemic acute infectious disease, especially with the features of possible asymptomatic carriers and high contagiousness. It causes acute respiratory distress syndrome and results in a high mortality rate if pneumonia is involved. Currently, it is difficult to quickly identify asymptomatic cases or COVID-19 patients with pneumonia due to limited access to reverse transcription-polymerase chain reaction (RT-PCR) nucleic acid tests and CT scans, which facilitates the spread of the disease at the community level, and contributes to the overwhelming of medical resources in intensive care units. Goal This study aimed to develop a scientific and rigorous clinical diagnostic tool for the rapid prediction of COVID-19 cases based on a COVID-19 clinical case database in China, and to assist global frontline doctors to efficiently and precisely diagnose asymptomatic COVID-19 patients and cases who had a false-negative RT-PCR test result. Methods With online consent, and the approval of the ethics committee of Zhongshan Hospital Fudan Unversity (approval number B2020-032R) to ensure that patient privacy is protected, clinical information has been uploaded in real-time through the New Coronavirus Intelligent Auto-diagnostic Assistant Application of cloud plus terminal (nCapp) by doctors from different cities (Wuhan, Shanghai, Harbin, Dalian, Wuxi, Qingdao, Rizhao, and Bengbu) during the COVID-19 outbreak in China. By quality control and data anonymization on the platform, a total of 3,249 cases from COVID-19 high-risk groups were collected. These patients had SARS-CoV-2 RT-PCR test results and chest CT scans, both of which were used as the gold standard for the diagnosis of COVID-19 and COVID-19 pneumonia. In particular, the dataset included 137 indeterminate cases who initially did not have RT-PCR tests and subsequently had positive RT-PCR results, 62 suspected cases who initially had false-negative RT-PCR test results and subsequently had positive RT-PCR results, and 122 asymptomatic cases who had positive RT-PCR test results, amongst whom 31 cases were diagnosed. We also integrated the function of a survey in nCapp to collect user feedback from frontline doctors. Findings We applied the statistical method of a multi-factor regression model to the training dataset (1,624 cases) and developed a prediction model for COVID-19 with 9 clinical indicators that are fast and accessible: 'Residing or visiting history in epidemic regions', 'Exposure history to COVID-19 patient', 'Dry cough', 'Fatigue', 'Breathlessness', 'No body temperature decrease after antibiotic treatment', 'Fingertip blood oxygen saturation<=93%', 'Lymphopenia', and 'C-reactive protein (CRP) increased'. The area under the receiver operating characteristic (ROC) curve (AUC) for the model was 0.88 (95% CI: 0.86, 0.89) in the training dataset and 0.84 (95% CI: 0.82, 0.86) in the validation dataset (1,625 cases). To ensure the sensitivity of the model, we used a cutoff value of 0.09. The sensitivity and specificity of the model were 98.0% (95% CI: 96.9%, 99.1%) and 17.3% (95% CI: 15.0%, 19.6%), respectively, in the training dataset, and 96.5% (95% CI: 95.1%, 98.0%) and 18.8% (95% CI: 16.4%, 21.2%), respectively, in the validation dataset. In the subset of the 137 indeterminate cases who initially did not have RT-PCR tests and subsequently had positive RT-PCR results, the model predicted 132 cases, accounting for 96.4% (95% CI: 91.7%, 98.8%) of the cases. In the subset of the 62 suspected cases who initially had false-negative RT-PCR test results and subsequently had positive RT-PCR results, the model predicted 59 cases, accounting for 95.2% (95% CI: 86.5%, 99.0%) of the cases. Considering the specificity of the model, we used a cutoff value of 0.32. The sensitivity and specificity of the model were 83.5% (95% CI: 80.5%, 86.4%) and 83.2% (95% CI: 80.9%, 85.5%), respectively, in the training dataset, and 79.6% (95% CI: 76.4%, 82.8%) and 81.3% (95% CI: 78.9%, 83.7%), respectively, in the validation dataset, which is very close to the published AI model. The results of the online survey 'Questionnaire Star' showed that 90.9% of nCapp users in WeChat mini programs were 'satisfied' or 'very satisfied' with the tool. The WeChat mini program received a significantly higher satisfaction rate than other platforms, especially for 'availability and sharing convenience of the App' and 'fast speed of log-in and data entry'. Discussion With the assistance of nCapp, a mobile-based diagnostic tool developed from a large database that we collected from COVID-19 high-risk groups in China, frontline doctors can rapidly identify asymptomatic patients and avoid misdiagnoses of cases with false-negative RT-PCR results. These patients require timely isolation or close medical supervision. By applying the model, medical resources can be allocated more reasonably, and missed diagnoses can be reduced. In addition, further education and interaction among medical professionals can improve the diagnostic efficiency for COVID-19, thus avoiding the transmission of the disease from asymptomatic patients at the community level.
- Downloaded 378 times
- Download rankings, all-time:
- Site-wide: 87,925
- In infectious diseases: 4,236
- Year to date:
- Site-wide: 44,725
- Since beginning of last month:
- Site-wide: 29,183
Downloads over time
Distribution of downloads per paper, site-wide
- 27 Nov 2020: The website and API now include results pulled from medRxiv as well as bioRxiv.
- 18 Dec 2019: We're pleased to announce PanLingua, a new tool that enables you to search for machine-translated bioRxiv preprints using more than 100 different languages.
- 21 May 2019: PLOS Biology has published a community page about Rxivist.org and its design.
- 10 May 2019: The paper analyzing the Rxivist dataset has been published at eLife.
- 1 Mar 2019: We now have summary statistics about bioRxiv downloads and submissions.
- 8 Feb 2019: Data from Altmetric is now available on the Rxivist details page for every preprint. Look for the "donut" under the download metrics.
- 30 Jan 2019: preLights has featured the Rxivist preprint and written about our findings.
- 22 Jan 2019: Nature just published an article about Rxivist and our data.
- 13 Jan 2019: The Rxivist preprint is live!