As first reported in a paper from researchers led by the Massachusetts Institute of Technology, there were claims that AI could detect the difference in cough sounds between those with and without COVID-19 with up to 98.5 percent accuracy, reports The Register.
The result led to efforts to build an app powered by the algorithms to provide people with a cheap and easy method to test for the novel coronavirus.
Cough in a Box Technology to Detect COVID-19
The Department of Health and Social Care in the UK even went so far as to award Fujitsu two contracts totaling more than 1,00,000 pounds to develop the government’s so-called “Cough In A Box” initiative in 2021, according to the report.
The software would collect audio recordings of coughs from users to analyze on its Covid-19 app.
Researchers from The Alan Turing Institute and Royal Statistical Society, commissioned by the UK Health Security Agency, conducted an independent review of audio-based AI tech as a COVID-19 screening tool.
Is Detecting COVID-19 Through Cough Sounds Effective?
They found that even the most accurate cough-detecting model outperformed a model based on user-reported systems and demographic data like age and gender.
The researchers examined data from over 67,000 people recruited through the National Health Service’s Test and Trace and REACT-1 programmes, which asked participants to send back COVID-19 nose and throat swab test results as well as recordings of them coughing, breathing, and talking.
In an attempt to see whether coughs could be used as a biomarker, the researchers trained an AI model based on audio recordings and test results, said the report.
“But as we continued to analyze the results, it appeared that the accuracy was likely due to an effect in statistics called confounding — where models learn other variables which correlate with the true signal, as opposed to the true signal itself,” Kieran Baker, a statistics PhD student at King’s College London and research assistant at the Alan Turing Institute, was quoted as saying.
The confounding was caused by the Test and Trace system’s recruitment bias, which required participants to have at least one symptom in order to participate. The researchers conducted additional tests, pairing participants of similar ages and genders, with only one having COVID-19, the report mentioned.
“When we evaluated these models on the matched data, the models failed to perform well, and so we conclude that our models cannot detect a COVID-19 bio-acoustic marker from this data,” Baker said.