A brief introduction to concepts and applications of artificial intelligence in dental imaging

(1)

https://doi.org/10.1007/s11282-020-00468-5 RAPID COMMUNICATION

A brief introduction to concepts and applications of artificial intelligence in dental imaging

Ruben Pauwels¹

Received: 12 February 2020 / Accepted: 18 July 2020 / Published online: 16 August 2020

Abstract

This report aims to summarize the fundamental concepts of Artificial Intelligence (AI), and to provide a non-exhaustive overview of AI applications in dental imaging, comprising diagnostics, forensics, image processing and image reconstruction.

AI has arguably become the hottest topic in radiology in recent years owing to the increased computational power available to researchers, the continuing collection of digital data, as well as the development of highly efficient algorithms for machine learning and deep learning. It is now feasible to develop highly robust AI applications that make use of the vast amount of data available to us, and that keep learning and improving over time.

Keywords Artificial intelligence · Machine learning · Deep learning · Dentistry · Radiology

Introduction

Due to the recently increased development of artificial intelligence (AI) applications in all branches of society, it has now become a hot topic in radiology [1], seeing that both industry, academia and clinicians are recognizing the tremendous potential of AI methods applied to medical images.

AI is more of a conceptual term, comprising the ability of machines to mimic human cognition and behavior. Machine learning (ML) and deep learning (DL) are implementations of computational methods (i.e., algorithms) that analyze data and induce models describing certain properties of this data, allowing for future predictions on new sets of data.

Thus, ML and DL can be defined as computational systems that learn over time based on experience. The distinction between ML and DL can be made based on the complexity of the network that is used to extract features from the data;

in turn, this is related to the complexity of the data itself.

For regression or classification of simple numerical data, ML typically suffices; for analyzing large sets of imaging or other complex data, DL is warranted. Therefore, the current focus in medical research, especially in radiology, is

primarily on DL applications. A particular type of DL based on convolutional neural networks (CNNs) has shown wide applicability on imaging data, as it can be used to extract a wide array of features by passing an input image through several filtering layers, conceptually similar to how the human visual system works.

How ML/DL systems ‘learn’

The basic concept of ML/DL is that input data (e.g., numbers, text, images) are coupled to a particular type of output, which can also be of any shape or form, e.g., a yes/no answer, a categorical description, an image, etc. Input and output are connected through a model that provides the best possible prediction of an output for any given input. The model is established in several repeated steps, which is the

‘learning’ aspect of ML/DL; by adjusting the weights in the model and evaluating the expected vs. actual output during each cycle, the predictive power of the model improves. A distinction can be made between supervised, reinforcement or unsupervised learning [2]. In supervised learning (Fig. 1, which is the most relevant type for radiological applications, the ML/DL model is ‘trained’ using a dataset that is labeled;

in other words, the expected output is known and the algorithm determines a model with the highest predictive power based on this training data. Unsupervised learning implies that the expected output is not known, but that there is a notion of having particular subgroups or correlations within

* Ruben Pauwels

pauwelsruben@hotmail.com

1 Aarhus Institute of Advanced Studies (AIAS), Aarhus University, Høegh-Guldbergs Gade 6B, 8000-C Aarhus, Denmark

(2)

154 Oral Radiology (2021) 37:153–160

complex datasets; applications in medicine are somewhat difficult to conceive, as there is typically always a particular task in mind (e.g., diagnosis), which would make supervised learning more suitable. Reinforcement learning is in-between supervised and unsupervised learning, as it is based on a reward system without prior knowledge by the AI system of the required output [3]. Typical applications in medicine involve situations where exact labeling of data is not feasible due to the absence of a ground truth, or when a pre-trained model using supervised learning undergoes further testing on new data under supervision of clinical experts that verify the model output.

It is expected that AI will evolve into an integral and essential part of a radiological workflow in numerous ways.

Each link in the imaging chain, i.e., image acquisition, image reconstruction and processing, radiological analysis and reporting, and storage can benefit from the computational power and versatility of AI. Already, several subfields of medical imaging have explored specific applications of AI [4–9]. The following section will focus on selected applications in dental radiology. It is not intended to cover all applications, nor all previously published work for any given application. For a more in-depth review of the currently available literature, refer to the overview by Hwang et al.

[10] and the systematic review by Hung et al. [11].

AI and DL in dental imaging

Diagnostics

One of the major potential roles of AI in radiology is in the diagnosis of pathology. It has been shown that a well-trained AI model could reach or even surpass the performance of human observers. It could therefore be considered for more obscure lesions that could easily be missed by a clinician, or as a ‘first pass’ analysis with the aim of saving radiological interpretation time by highlighting potential pathology that warrant further investigation.

A recent review by Hwang et al. identified 25 studies that used DL in dental imaging, with convolutional neural networks used as the predominant component of the DL model in all studies [10]. Whereas initial research in this field often involved a rather limited training dataset, they noted an increase in the median size of training data in recent studies, indicating the increasing efforts being made in recent years in this field.

Poedjiastoeti and Suebnukarn investigated the use of CNNs in the diagnosis of ameloblastomas and keratocystic odontogenic tumors on panoramic radiographs, visualiz- ing the output of the CNN as a heatmap that flags potential regions of interest (Fig. 2) [12]. The sensitivity, specificity and accuracy were nearly identical between the CNN and oral and maxillofacial surgeons, whereas the ‘interpretation time’ was 38 s for the former vs 23.1 min for the latter.

Murata et al. evaluated CNNs for the detection of maxillary sinusitis on panoramic radiographs [13]. The AI’s performance was slightly worse compared with experienced radiologists, but considerably higher than that of dental residents (Fig. 3).

An application for which ML and DL may result in vastly superior performance compared with clinical observers is the early detection of osteoporosis. This could be done indirectly by extracting a large series of features from an image (e.g., cortical width, fractal dimension on a panoramic radiographs) and combining this with demographic data and other patient-specific factors to predict the onset of

Fig. 1 Supervised vs. unsupervised vs. reinforcement learning. Top:

in supervised learning, the data are labeled (e.g., presence/absence of lesion), and used for training a machine/deep learning (ML/DL) model. Middle: unsupervised learning is typically applied for cluster- ing, in which a set of unlabeled data are split up into groups using ML or DL. Bottom: reinforcement learning involves a feedback loop, in which predictions from a, typically pre-trained, ML/DL model are verified by human experts

(3)

osteoporosis (in which ML suffices). Alternatively, through the use of CNNs or other algorithms, abstract features can be extracted from an image (DL). Using the former approach, Kavitha et al. developed a classification system that resulted in an accuracy of 96% vs. lumbar spine bone mineral density (BMD) and 98.9% vs. femoral neck BMD [14]. Using the latter approach, Chu et al. showed an overall accuracy of 90% for a DL network that combined 8 regions of interest on panoramic radiographs [15].

Other studies have focused on the diagnosis of pathology on intra-oral radiographs. Applying CNNs in the detection of periodontally compromised teeth (PCT) on 1740 periapical radiographic images, for premolars and molars, the total diagnostic accuracy was 81.0% and 76.7%, respectively, showing the highest diagnostic accuracy for severe PCT (Fig. 4) [16]. Krois et al. evaluated CNNs for the evaluation of periodontal bone loss, showing a lower sensitivity, but higher specificity compared with dentists (Fig. 5) [17].

Fig. 2 Convolutional neural networks for tumor diagnosis on panoramic radiographs. Heat maps indicate regions marked by the AI as having potential pathology in red/yellow. Left: patient with multi- locular cystic radiolucency at the right angle of the mandible. Right:

patient with a unilocular cystic radiolucency at the left angle of the mandible. Reproduced from Poedjiastoeti and Suebnukarn under a Creative Commons Attribution Non-Commercial License [12]

Fig. 3 Left: regions of interest used for AI-based detection of maxillary sinusitis on panoramic radiographs. Right: comparative diagnostic performance of the AI system vs. experienced radiologists and

dental residents. Adapted from Murata et al. with permission by the Japanese Society for Oral and Maxillofacial Radiology and Springer Nature Singapore [13]

(4)

156 Oral Radiology (2021) 37:153–160

Studies focusing on AI-based diagnosis in dental CT and CBCT are scarce as of yet, which could be attributed to the higher complexity of 3D images as well as practi- cal difficulties in gathering a large training dataset and appropriate labels (i.e., ground truth used during training).

Johari et al. applied a neural network to both intra-oral and CBCT images for the diagnosis and vertical root fractures, finding a high sensitivity for both modalities but a much higher specificity for CBCT [18]. Ariji et al. used a CNN for the detection of cervical lymph node metastasis in contrast-enhanced CT images of oral cancer patients, showing no significant difference with the performance of radiologists (CNN AUC 0.80, radiologists AUC 0.83) [19], however, it can be assumed that the time for radiological reporting was much shorter for the CNN than for the radiologists.

Forensics

Forensic analysis is ideally suited for AI, owing to the feasi- bility of having vast databases of labeled data available for the training of neural networks or other types of ML and DL techniques. Several forensic applications can be conceived.

In the context of age estimation, De Tobel et al. explored the use of CNNs in the staging of mandibular third molars on panoramic radiographs (Fig. 6), showing a mean accuracy of 0.51 and a mean absolute difference with the ground truth of 0.6 stages [20]. Indirect (through staging) or direct age estimation accuracy using AI is expected to increase considerably as CNN-based techniques are applied on large datasets. Other forensic applications include, but are not limited to, matching post-mortem images with medical records, reconstructing facial features based on imaging and genetic

Fig. 4 Detection of periodontally compromised teeth (PCT) using convolutional neural networks. Reproduced from Lee et al. under a Creative Commons Attribution Non-Commercial License [16]

Fig. 5 AI-based detection of periodontal bone loss on intra-oral radiographs. Top row: selected true positive radiographs. Bottom row:

selected true negative radiographs. Numbers above each image depict

the probability (0–1) of bone loss as determined by the AI. Adapted from Krois et al. under a Creative Commons Attribution 4.0 Interna- tional License [17]

(5)

data, and detecting and analyzing potential trauma or other aberrations.

Image processing

AI can serve an important role in types of image processing that are cumbersome or time-consuming for human observers, or for which other automatic processing methods show inadequate performance. A principal example is the segmentation of 3D models of bones and/or teeth on CBCT scans for surgical planning. Segmentation accuracy in CBCT using conventional thresholding is severely affected by the instability of grey values, which cannot be reliably calibrated as Hounsfield Units [21]. Using ML/

DL techniques, this limitation can be overcome and accurate, fast and robust bone/teeth segmentation can become

feasible. One challenge in the training of models for segmentation is having highly accurate ground truth data.

Another type of segmentation is landmark identification on 2D or 3D images, which could be facilitated using a well- trained DL model [22]. Applications in dentistry include tooth numbering on intra-oral radiographs (Fig. 7) [23]

and panoramic radiography [24]. Furthermore, automated cephalometric landmark identification has been researched extensively [11], and is likely to become the first widely used dental ‘AI’ application in clinical practice. Further- more, with the increasing use of 3D and multi-modal imaging (e.g., CT, MRI, nuclear medicine and molecular imaging), the use of DL in image registration has been explored [25] in order to match anatomical, diagnostic and functional information, or to match images acquired at different time points (e.g., pre- and post-operative).

Fig. 6 Using convolutional neural networks for tooth staging. Left:

ten stages of human third molar development used in forensics. Right:

pre-processing of panoramic radiograph by aligning the long axis and

determining a bounding box, which serves as input for a convolutional neural network. Adapted from De Tobel et al. with the author’s permission [20]

Fig. 7 Sample images of challenging test cases that were correctly annotated by neural networks: (c1) implant restoration, (c2) restored crown and bridge, (c3) teeth without crowns. Adapted from Chen et al. under a Creative Commons Attribution 4.0 International License [23]

(6)

158 Oral Radiology (2021) 37:153–160

Image reconstruction

Deep learning has shown great potential in improving CT (or CBCT) reconstruction. Strictly speaking, this is not an AI application, as the DL model does not replace a human func- tion. Currently, the most common reconstruction technique is based on filtered backprojection (FBP), which is fast but prone to several reconstruction errors that lead to artifacts.

Alternative reconstruction techniques, under the group name iterative reconstruction (IR), can incorporate artifact reduction and provide excellent image quality even at very low doses, but can be excessively time-consuming. Deep learning can be incorporated either in FBP or IR, for example by pre-processing the RAW data or by post-processing a reconstructed scan. Depending on the implementation and the nature of the training data, DL can be applied for noise reduction, artifact reduction, or other image quality enhance- ments (Fig. 8) [26]. One of the main benefits of DL over advanced IR is that the former does not significantly increase reconstruction time, as a trained (neural/other) network can be applied to RAW or reconstructed data at a very high data throughput. The main challenge for this application is the generation of suitable training data, which may require accurate CT simulation [27] or the acquisition of high-dose reference scans [26].

Conclusion

AI in dentistry, using ML and/or DL, has shown tremendous promise for a wide variety of applications. It can be expected that AI will become part of the clinician’s toolset in the very near future, although this will typically require extensive pre-clinical and clinical research to ensure the robustness of the AI models. Furthermore, responsibilities regarding the ethical aspects of AI need further discussion between the different stakeholders. Specific issues that should be addressed are:

1. The handling and sharing of large sets of (clinical) training data by researchers and AI developers, which should adhere to current local regulations on data protection (e.g., European Union’s General Data Protection Regu- lation 2016/679).

2. The approval of newly developed AI tools for clinical testing and commercial distribution by local governmen- tal bodies, which should undergo the same scrutiny as any other diagnostic test (e.g., CE marking as Class IIb Medical Device)

3. Revisiting the responsibilities of the different role-play- ers in the diagnostic process. In case of misdiagnosis involving an AI tool, the blame could be passed to the AI developer, the oral radiologist (if applicable) and/or the

dental practitioner. It is more than likely that a separate set of unambiguous regulations need to be developed to define responsibilities in the diagnostic use of AI.

A recent joint statement by several European and North American societies addresses the ethical use of AI in radiology in more depth [28]. The key points were the maximization of benefit at minimal harm, transparency, continued accountability of human stakeholders, and the need to develop codes of ethics and practice. Due to several unique aspects of dentistry (e.g., wide array of dental treatments with varying potential morbidity, oral radiology not recognized as a specialty by several countries), it can be recommended that dental associations develop a specific statement regarding the current and future use of AI in dentistry.

Fig. 8 Artifact correction in low-dose dental CT. Left (a, b, c): origi- nal scan. Right (d, e, f): after processing using a type of generative adversarial network. Arrows highlight the improved depiction of fine anatomical details. Processing time was 0.18 s per image. Adapted from Hu et al. with permission from John Wiley and Sons [25]

(7)

Acknowledgements R. Pauwels is supported by the European Union Horizon 2020 Research and Innovation Programme under the Marie Skłodowska-Curie Grant agreement number 754513 and by Aarhus University Research Foundation (AIAS-COFUND).

Compliance with ethical Standards

Conflict of interest The author declares no conflict of interest.

Human and animal rights This article does not contain any studies with human or animal subjects performed by any of the authors.

References

1. Recht M, Bryan RN. Artificial Intelligence: threat or boon to radiologists? J Am Coll Radiol. 2017;14:1476–80. https ://doi.

org/10.1016/j.jacr.2017.07.007.

2. Kohli M, Prevedello LM, Filice RW, Geis JR. Implement- ing machine learning in radiology practice and research. AJR Am J Roentgenol. 2017;208:754–60. https ://doi.org/10.2214/

AJR.16.17224 .

3. Clarke AM, Friedrich J, Tartaglia EM, Marchesotti S, Senn W, Herzog MH. Human and machine learning in non-Markovian decision making. PLoS One. 2015;10:e0123105. https ://doi.

org/10.1371/journ al.pone.01231 05.

4. Plis SM, Hjelm DR, Salakhutdinov R, Allen EA, Bockholt HJ, Long JD, et al. Deep learning for neuroimaging: a validation study. Front Neurosci. 2014;8:229. https ://doi.org/10.3389/fnins .2014.00229 .

5. Houssami N, Lee CI, Buist DSM, Tao D. Artificial intelligence for breast cancer screening: opportunity or hype? Breast. 2017;36:31–

3. https ://doi.org/10.1016/j.breas t.2017.09.003.

6. Lakhani P, Sundaram B. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology. 2017;284:574–82. https ://doi.

org/10.1148/radio l.20171 62326 .

7. Lee H, Troschel FM, Tajmir S, Fuchs G, Mario J, Fintelmann FJ, et al. Pixel-level deep segmentation: artificial intelligence quantifies muscle on computed tomography for body morpho- metric analysis. J Digit Imaging. 2017;30:487–98. https ://doi.

org/10.1007/s1027 8-017-9988-z.

8. Olczak J, Fahlberg N, Maki A, Razavian AS, Jilert A, Stark A, et al. Artificial intelligence for analyzing orthopedic trauma radiographs. Acta Orthop. 2017;88:581–6. https ://doi.

org/10.1080/17453 674.2017.13444 59.

9. Prevedello LM, Erdal BS, Ryu JL, Little KJ, Demirer M, Qian S, et al. Automated critical test findings identification and online notification system using artificial intelligence in imaging. Radiol- ogy. 2017;285:923–31. https ://doi.org/10.1148/radio l.20171 62664 10. Hwang JJ, Jung YH, Cho BH, Heo MS. An overview of deep .

learning in the field of dentistry. Imaging Sci Dent. 2019;49:1–7.

https ://doi.org/10.5624/isd.2019.49.1.1.

11. Hung K, Montalvao C, Tanaka R, Kawai T, Bornstein MM.

The use and performance of artificial intelligence applications in dental and maxillofacial radiology: a systematic review. Den- tomaxillofac Radiol. 2020;49:20190107. https ://doi.org/10.1259/

dmfr.20190 107.

12. Poedjiastoeti W, Suebnukarn S. Application of convolutional neural network in the diagnosis of jaw tumors. Healthc Inform Res.

2018;24:236–41. https ://doi.org/10.4258/hir.2018.24.3.236.

13. Murata M, Ariji Y, Ohashi Y, Kawai T, Fukuda M, Funakoshi T, et al. Deep-learning classification using convolutional neural network for evaluation of maxillary sinusitis on panoramic radiography. Oral Radiol. 2018;35:301–7. https ://doi.org/10.1007/s1128 2-018-0363-7.

14. Kavitha MS, Ganesh Kumar P, Park SY, Huh KH, Heo MS, Kurita T, et al. Automatic detection of osteoporosis based on hybrid genetic swarm fuzzy classifier approaches. Dentomaxillofac Radiol. 2016;45:20160076. https ://doi.org/10.1259/dmfr.20160 15. Chu P, Bo C, Liang X, Yang J, Megalooikonomou V, Yang F, 076.

Huang B, Li X, Ling H. Using octuplet siamese network for osteoporosis analysis on dental panoramic radiographs. Conf Proc IEEE Eng Med Biol Soc. 2018;2018:2579–82. https ://doi.

org/10.1109/EMBC.2018.85127 55.

16. Lee JH, Kim DH, Jeong SN, Choi SH. Diagnosis and prediction of periodontally compromised teeth using a deep learning-based convolutional neural network algorithm. J Periodontal Implant Sci. 2018;48:114–23. https ://doi.org/10.5051/jpis.2018.48.2.114.

17. Krois J, Ekert T, Meinhold L, Golla T, Kharbot B, Wittemeier A, et al. Deep learning for the radiographic detection of periodontal bone loss. Sci Rep. 2019;9:8495. https ://doi.org/10.1038/s4159 8-019-44839 -3.

18. Johari M, Esmaeili F, Andalib A, Garjani S, Saberkari H. Detec- tion of vertical root fractures in intact and endodontically treated premolar teeth by designing a probabilistic neural network: an ex vivo study. Dentomaxillofac Radiol. 2017;46:20160107. https ://doi.org/10.1259/dmfr.20160 107.

19. Ariji Y, Fukuda M, Kise Y, Nozawa M, Yanashita Y, Fujita H, et al. Contrast-enhanced computed tomography image assessment of cervical lymph node metastasis in patients with oral cancer by using a deep learning system of artificial intelligence. Oral Surg Oral Med Oral Pathol Oral Radiol. 2019;127:458–63. https ://doi.

org/10.1016/j.oooo.2018.10.002.

20. De Tobel J, Radesh P, Vandermeulen D, Thevissen PW. An automated technique to stage lower third molar development on panoramic radiographs for age estimation: a pilot study. J Forensic Odontostomatol. 2017;2:42–544.

21. Pauwels R, Jacobs R, Singer SR, Mupparapu M. CBCT-based bone quality assessment: are Hounsfield units applicable? Den- tomaxillofac Radiol. 2015;44:20140238. https ://doi.org/10.1259/

dmfr.20140 238.

22. Zhang J, Liu M, Wang L, Chen S, Yuan P, Li J, et al. Joint cra- niomaxillofacial bone segmentation and landmark digitization by context-guided fully convolutional networks. Med Image Comput Comput Assist Interv. 2017;10434:720–8. https ://doi.

org/10.1007/978-3-319-66185 -8_81.

23. Chen H, Zhang K, Lyu P, Li H, Zhang L, Wu J, et al. A deep learning approach to automatic teeth detection and numbering based on object detection in dental periapical films. Sci Rep. 2019;9:3840.

https ://doi.org/10.1038/s4159 8-019-40414 -y.

24. Tuzoff DV, Tuzova LN, Bornstein MM, Krasnov AS, Kharch- enko MA, Nikolenko SI, et al. Tooth detection and numbering in panoramic radiographs using convolutional neural networks. Den- tomaxillofac Radiol. 2019;48:20180051. https ://doi.org/10.1259/

dmfr.20180 051.

25. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoo- rian M, et al. A survey on deep learning in medical image analysis.

Med Image Anal. 2017;42:60–88. https ://doi.org/10.1016/j.media .2017.07.005.

26. Hu Z, Jiang C, Sun F, Zhang Q, Ge Y, Yang Y, et al. Artifact correction in low-dose dental CT imaging using Wasserstein generative adversarial networks. Med Phys. 2019;46:1686–96. https ://

doi.org/10.1002/mp.13415 .

27. Pauwels R, Oliveira-Santos C, Oliveira ML, Watanabe PCA, Araújo Faria V, Jacobs R, et al. Artefact reduction in cone beam

(8)

160 Oral Radiology (2021) 37:153–160 CT through deep learning: a pilot study using neural networks in

the projection domain. In: 22nd International congress of Den- toMaxilloFacial radiology, Philadelphia, PA, USA, 2019.

28. Geis JR, Brady AP, Wu CC, Spencer J, Ranschaert E, Jaremko JL, et al. Ethics of artificial intelligence in radiology: summary of the Joint European and North American Multisociety State- ment. Radiology. 2019;293:436–40. https ://doi.org/10.1148/radio l.20191 91586 .

Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.