Data Science Investment Counter —Funding raised by UK data science companies in 2018.
£ 5.640 Million

AI may be as effective as doctors at diagnosing disease

Artificial intelligence (AI) would be able to detect diseases from medical imaging with similar levels of accuracy as medical doctors, according to new research by the University of Birmingham and University Hospitals Birmingham NHS Foundation Trust.

The research, which was was published today in The Lancet Digital Health journal, represents the first systematic review and meta-analysis combining all the available evidence from the scientific literature on the matter.

The authors of the paper study warned that only a limited amount of high-quality studies that directly compare the performance of humans and machines were currently available in real clinical environments.

“We reviewed over 20,500 articles, but less than one per cent of these were sufficiently robust in their design and reporting that independent reviewers had high confidence in their claims,” said Alastair Denniston, professor at the University of Birmingham and at the University Hospitals Birmingham NHS Foundation Trust.

“What’s more, only 25 studies validated the AI models externally using medical images from a different population, and just 14 studies actually compared the performance of AI and health professionals using the same test sample.”

Because of this, the real diagnostic power of deep learning remains uncertain, as well as the possibilities of fully validating AI’s performance in this field. The researchers have therefore called for higher standards of research in order to improve future investigations on the topic.

The paper also clarified that, while AI was able to rival human results in providing medical diagnosing, it was far from being the undisputed winner in every or even most cases.

“Within those handful of high-quality studies, we found that deep learning could indeed detect diseases ranging from cancers to eye diseases as accurately as health professionals. But it’s important to note that AI did not substantially out-perform human diagnosis.”

This would be due to the fact that few of the studies on the subject are done in real clinical environments, and that determining diagnostic accuracy requires high-quality comparisons in patients, not just datasets.

Poor reporting was another cause of AI failing to outperform healthcare clinicians, and most studies did not report missing data, which limited the conclusions that could be drawn.

“Evidence on how AI algorithms will change patient outcomes needs to come from comparisons with alternative diagnostic tests in randomised controlled trials,” said Dr Livia Faes from Moorfields Eye Hospital, London.

“So far, there are hardly any such trials where diagnostic decisions made by an AI algorithm are acted upon to see what then happens to outcomes which really matter to patients, like timely treatment, time to discharge from hospital, or even survival rates.”

Image via wanicon on flaticon.


Co-working space and blog dedicated to all things data science.

Subscribe to our newsletter