Pearl today announced the results of a landmark study comparing the diagnostic performance of three experienced human dentists to the performance of an artificial intelligence (AI) diagnostic system. The study evaluating agreement between human and AI analyses of a set of 8,767 bitewing and periapical radiographs found the AI system to be more consistent and accurate at predicting the presence of tooth decay.
In addition to validating the performance of AI diagnostic systems for dental radiology, the study exposed an unsettling lack of diagnostic consensus among human dentists. While they showed moderate alignment – 79% unanimous agreement – when it came to the absence of decay, the three human dentists unanimously agreed about the presence of decay in only 370 X-rays––just 4.2% of the total. In nearly one in five instances, even when two dentists identified decay in an X-ray, the third dentist did not. The conspicuous diagnostic variation across such a massive radiographic data set is concerning.
“Our intention in producing this study was simply to demonstrate the efficacy of computer vision machine learning diagnostics in dental radiology, but the secondary findings dredge up a major deficiency in dental healthcare,” said Pearl CEO Ophir Tanz. “While diagnostic inconsistency may be a natural byproduct of human professionals making case-by-case judgements, at scale inconsistency in the standard of care results in suboptimal patient treatment with broader implications for population health at large. Fortunately, the study’s findings also point to a solution: The machine’s diagnostic performance shows that AI is capable of infusing consistency into the bedrock of dental care.”
The superior accuracy of the AI system’s diagnostic conclusions remained consistent both when any single dentist’s diagnosis was held as ‘ground truth’, and when ground truth came from two dentists. The AI also tended to agree more with the human dentists than the human dentists did with each other, a product of the AI’s greater sensitivity to potential decay in X-rays.
As a machine learning system, the AI’s intelligence comes from studying millions of X-rays evaluated by hundreds of human dentists. Because the AI is bringing insights from all those dentists to bear on its analysis., when decay is present, the AI is more likely to identify it.
Acknowledging the study’s limits, the authors say they would like future studies to evaluate the accuracy of more precise diagnostic conclusions, like level of decay, as well as expand the number of human participants.
Dr. Sanjay Mallya, Chair and Associate Professor in the Section of Oral and Maxillofacial Radiology in the Division of Diagnostic and Surgical Sciences at the UCLA School of Dentistry, agrees.
“More studies like this are in order,” Dr. Mallya said. “It would be useful, for instance, to drill deeper to see if the discrepancy evident in this study persists for caries across varying degrees of severity. Does the discrepancy exist between caries which only impact enamel, dentin, etc? The more data we have to give credence to the merits of AI diagnostics the better, because it is this sort of data that will help doctors embrace AI as a tool for bringing greater excellence and confidence to their patient care.”
Until that happens, the study published today stands as the first truly compelling data-driven glimpse into a brave new dental future.
To read the complete study – Can a Computer Identify Carious Lesions in Dental X-Rays As Accurately As Humans? An exploratory study comparing diagnostic assessments performed by humans and a specialized computer vision system – please visit hellopearl.com/insights.
