COVID-19 diagnosis and the patient’s prognosis can be predicted from chest CT-scans, X-rays or sound recordings of coughs or breathing.

ML methods offer very different ways to help confining the spread of infectious diseases, e.g., in developing vaccines and drugs for the treatment of COVID-19. Especially in developing countries with limited testing capacities, the ML enhanced tests can evolve into an efficient tool in combating a pandemic. Routine screening of the blood tests would allow frequent, fast and broad testing at low cost, thus providing a powerful tool to reduce new outbreaks in the hospital. The routine blood tests are acquired anyway, therefore, no additional efforts are caused by screening with ML models. Instead of RT-PCR tests or antigen tests, routine blood tests can be automatically scanned for COVID-19: machine learning (ML) models can predict the diagnoses on the basis of blood tests, which are taken in the routine processes of the hospital. In contrast, antigen tests are cheap and fast, but they come with considerably lower sensitivity than RT-PCR tests. However, RT-PCR tests are expensive, time-consuming, and not suited for high-throughput or large-scale testing efforts. Reverse transcription polymerase chain reaction (RT-PCR) remains the gold standard test for the coronavirus disease 2019 (COVID-19). Therefore, frequent re-training and re-assessment are indispensable for robust models enabling clinical utility. We present the novel finding that domain shifts strongly affect machine learning models for COVID-19 diagnosis and deteriorate their predictive performance and credibility. We reveal domain shifts by evaluating the models on a large-scale dataset with different assessment strategies, such as temporal validation. We further set out to estimate the mortality risk based on routinely acquired blood tests in a hospital setting throughout pandemics and under domain shifts. We investigate whether domain shifts are present in COVID-19 datasets and how they affect machine learning methods. Therefore, machine learning models for diagnosing COVID-19 or other diseases may not be reliable and degrade in performance over time.

Domain shifts can be caused, e.g., by changes in the disease prevalence (spreading or tested population), by refined RT-PCR testing procedures (way of taking samples, laboratory procedures), or by virus mutations. However, we hypothesize that changes in the underlying distribution of the data, so called domain shifts, affect the predictive performance and reliability and are a reason for the failure of such machine learning models in clinical application. Many previous studies claim to have developed machine learning models that diagnose COVID-19 from blood tests.

Chemlab primal contact