Classification of presence of missing teeth in each quadrant using deep learning artificial intelligence on panoramic radiographs of pediatric patients

1. Introduction

Teeth, which are important in human life, can be lost due to congenitally missing, traumas, caries, orthodontic issues and other factors [1, 2]. Tooth loss can lead to a range of problems, including loss of space, midline deviation and movement of adjacent teeth in adults [3]. In pediatric patients, complications may be more severe due to ongoing skeletal and alveolar bone growth. Tooth loss at this age can negatively affect the quality of life, causing reduced masticatory efficiency, malocclusion and speech problems [1, 4]. Therefore, timely and appropriate treatment is necessary to manage tooth loss in pediatric patients. Additionally, early and precise diagnosis holds paramount importance in attaining effective and efficient outcomes [5].

Dentists may use various methods for oral examinations in pediatric patients. Among these methods, X-ray imaging is commonly used, adhering to the principle of “as low as reasonably achievable (ALARA)” [6]. In particular, panoramic radiography provides valuable information about various anatomical structures, including the teeth, maxillary sinuses and nasal septum. Therefore, panoramic images are essential for accurate diagnosis of supernumerary, missing and impacted teeth [7, 8, 9]. However, panoramic radiographs can show overlapping structures and distortions owing to imaging techniques, patient positioning and technical issues [7]. Therefore, diagnostic accuracy can vary depending on the clinician’s expertise in interpreting panoramic radiographs [10, 11].

Recent advances in artificial intelligence (AI) have demonstrated its ability to learn and execute complex tasks [12]. Deep learning is a prominent subfield of AI, wherein an algorithm is designed to automatically obtain the desired outcomes through training with given pieces of information [13, 14]. Deep learning can autonomously perform information recognition, judgment and classification from given data [13]. Additionally, it can achieve high accuracy in discovering specific patterns, making it widely used in image and video analysis [13, 15, 16].

The region to be learned and evaluated using deep learning in radiographic images is referred to as the region of interest (ROI). To achieve high accuracy in deep learning algorithm systems, utilizing abundant high-quality training data and minimizing the ROI size proves to be beneficial [17, 18]. Larger input data sizes can result in longer training times, higher memory consumption, and the inclusion of irrelevant information outside the ROI, potentially causing confusion [19]. Therefore, setting the ROI to the smallest size possible while including the necessary information for analysis can increase training efficiency and enhance the accuracy of the results [10, 11, 20, 21].

Recently, deep learning has been applied in various fields, such as internet searches, gaming and medicine. In dentistry, deep learning is primarily used for the recognition and interpretation of radiographic images [15, 18, 20, 21, 22, 23]. Since 2021, studies conducted by Ahn et al. [20] and Park et al. [22] have demonstrated automated detection of supernumerary teeth and implant regions using deep learning in panoramic radiographs. Owing to the popularity of implant treatment in adults, there is growing interest in utilizing deep learning for the automatic detection of missing teeth in adult patients using panoramic or cone-beam computed tomography (CBCT) images [22, 24, 25]. However, the use of deep learning for radiographic image analysis in pediatric patients has been relatively limited. Currently, studies utilizing deep learning algorithms for the automated classification of missing teeth in pediatric patients are rare.

Therefore, this study aimed to develop a deep learning model for classifying missing teeth on panoramic radiographs in pediatric patients and evaluate the performance. Additionally, a novel method was proposed to enhance the efficiency of deep learning training by devising a new approach for setting the ROI in panoramic radiographs.

2. Materials and methods

2.2 Methods

2.2.1 Panoramic radiograph analysis

Panoramic radiographs were obtained using a Proline XC machine (Planmeca Proline XC, Planmec Co., Helsinki, Finland). After anonymization, the images were saved in a joint photographic experts group (JPEG) format. A dentist diagnosed the presence or absence of permanent missing teeth, excluding the third molars, on the entire panoramic radiograph. The diagnosis was made according to the Federation Dentaire Internationale (FDI) system. The presence (1) or absence (0) of missing teeth was recorded for each tooth in the 11–17, 21–27, 31–37 and 41–47 regions using Excel 2016 (Microsoft, Redmond, WA, USA).

2.2.2 Image pre-processing

In this study, a single panoramic radiograph was divided into four quadrants to minimize the size of the region of interest (ROI). The matrix laboratory (MATLAB) 2022b software (MathWorks Inc., Natick, MA, USA) was used to set the ROI. The upper right central incisor (#11), upper left central incisor (#21), lower left central incisor (#31), and lower right central incisor (#41) were labeled on the panoramic radiographs using the Image Labeler tool from the Computer Vision System Toolbox in MATLAB (Fig. 1a,b). To define each tooth, the mesio-incisal point angle, disto-incisal point angle, mesial cementoenamel junction, distal cementoenamel junction, mesial root apex, distal root apex, and midpoint between adjacent points were labeled [26]. Furthermore, the space between the upper and lower teeth was delineated following the method used in a previous study to set the posterior molar space [10]. For both the maxilla and mandible, the disto-occlusal point angle and the distal cemento-enamel junction of the most posterior molar, the alveolar ridge connecting posteriorly, the farthest point on the lower ramus, the cusp tip of the primary or permanent canines, and the midpoint between adjacent central incisors were labeled to establish the lines defining the interdental space (Fig. 1c).

2.2.3 Region of interest determination

To smoothen the shape, the interdental space labeled in MATLAB was subjected to a morphological closing operation in all directions using a disk-shaped element with a diameter of 300 pixels. Subsequently, dilation was performed using a disk-shaped element with a diameter of 40 pixels (Fig. 1d). The resulting shape was skeletonized and stretched to the farthest point of the mandibular ramus, forming the horizontal baseline (Fig. 1e). Within the range of the horizontal baseline, the upper part was designated as the maxilla, while the lower part represented the mandible (Fig. 1f). Information outside the regions of the maxilla and mandible was excluded from the ROI setting. Furthermore, a vertical baseline was set based on the midpoint between the central incisors (Fig. 1g). A rectangle was designed with the width representing the distance between the posterior point and the midpoint of the baseline and the height representing the distance from the midpoint to the root apex of the central incisor (Fig. 1h). To ensure that all teeth from one quadrant were included within a single rectangle, the width was increased by 1.2 times and the height by 1.5 times. One rectangle was created for each quadrant, resulting in a total of four rectangles (Fig. 1i). Only the information within the baseline-defined regions of the rectangles was retained (Fig. 1j). This process generated a dataset of 3224 samples from a total of 806 panoramic radiographs.

Fig. 1.Total flow of region of interest (ROI) setting. (a) Initial panoramic radiograph. (b) Labeling of anterior teeth. (c) Labeling of interdental space. (d) Dilation and smoothing of interdental space. (e) Skeletonization and stretch of interdental space, making horizontal baseline. (f) Setting the initial regions of the maxilla and mandible. (g) Exclusion of information outside the horizontal baseline and establishment of the vertical baseline. (h) Initial setting of ROI. (i) The final ROI with the width increased by 1.2 times and the height by 1.5 times. (j) Four final data for each quadrant from a single panoramic image.

2.2.4 Network architecture

To classify the presence of missing teeth, Inception-ResNet-V2, which has demonstrated high accuracy in analyzing dental radiographs using deep learning in previous studies, was employed [10, 15, 20].

2.2.5 Five-fold cross-validation and data augmentation

The final data were used for 5-fold cross-validation. 5-fold cross-validation allows all data to be utilized for both training and validation, thus reducing the errors caused by a limited amount of data. The dataset was randomly divided into five groups, with four groups used for training and one used for validation in each iteration. This process was repeated five times to ensure that all five groups were used as validation data (Fig. 2). Data augmentation techniques were applied to increase the size of the training dataset to prevent overfitting owing to the limited amount of data. These augmentation techniques included rotating the data between −8 and 8 degrees, translating the data horizontally or vertically between −15 and 15 pixels, scaling the data between 0.8 and 1.2 times horizontally or vertically, and flipping the data horizontally or vertically.

Fig. 2.The 5-fold cross-validation. The data was divided randomly into five groups, with four of them serving as the training sets and one as a validation set. ROI: Region of interest.

2.2.6 Training configuration

The deep learning network was trained using the Windows 10 operating system. MATLAB 2022b with a Deep Learning Toolbox and Parallel Computing Toolbox (MathWorks, Natick, MA, USA) was installed, and the training was performed on an NVIDIA Titan RTX GPU with an i7-8700K CPU and 32 GB of RAM running on Windows 10. The Adam optimizer was used to train the model for a maximum of 1500 epochs, with a mini-batch size of 16. The initial learning rate was set to e⁻⁴. If the validation accuracy did not increase by more than 30 times, the training process was terminated.

2.2.7 Diagnostic performance evaluation

The diagnostic performance of the best-performing model was evaluated using validation data. Sensitivity, specificity, accuracy, precision, recall, F1 score, receiver operating characteristic (ROC) curve, and area under the curve (AUC) were calculated to assess the model’s performance.

•

$Sensitivity=\frac{\mathrm{\text{TP}}}{TP+FN}$

•

$Specificity=\frac{\mathrm{\text{TN}}}{TN+FP}$

•

$Accuracy=\frac{TP+TN}{TP+TN+FN+FP}$

•

$Precision=\frac{\mathrm{\text{TP}}}{TP+FP}$

•

$Recall=\frac{\mathrm{\text{TP}}}{TP+FN}$

•

$F1Score=\frac{2\times (Recall\times Precision)}{Recall+Precision}$

• TP: true positive, FP: false positive, FN: false negative, TN: true negative.

2.2.8 Model visualization

The deep learning model was visualized using a gradient-weighted class activation mapping (Grad-CAM) technique. The CAM represented the region on which the AI primarily focused when making decisions about the presence or absence of missing teeth. The visualization depicted a color gradient from blue to red. The regions in blue indicated lower importance in the model’s decision-making process, whereas those in red indicated higher importance [20].

3. Results

3.1 Classification performance of deep learning model

The classification performance of the deep learning model is presented in Table 3. The average values for the accuracy, precision, recall, F1-score and AUC were 0.738, 0.730, 0.732, 0.731 and 0.730, respectively. The AUC values for each of the 5-fold cross-validation are shown in Fig. 3.

Table 3.Performance of missing teeth classification network through 5-fold cross-validation.

Pre-trained network (cv)	Sensitivity	Specificity	Accuracy	Precision	Recall	F1-score	AUC
1	0.645	0.784	0.716	0.708	0.709	0.708	0.708
2	0.639	0.844	0.762	0.754	0.757	0.756	0.754
3	0.607	0.801	0.737	0.728	0.731	0.730	0.728
4	0.690	0.807	0.746	0.739	0.740	0.739	0.739
5	0.596	0.832	0.729	0.720	0.723	0.721	0.720
Mean	0.635	0.814	0.738	0.730	0.732	0.731	0.730
AUC: Area under the curve.

Fig. 3.Receiver operating characteristic (ROC) curves of 5-fold cross-validation. (a) ROC of the first cross-validation. (b) ROC of the second cross-validation. (c) ROC of the third cross-validation. (d) ROC of the fourth cross-validation. (e) ROC of the fifth cross-validation. The area under the ROC curve represents the AUC value. AUC: Area under the curve.

3.2 Visualization of model classification

Areas that the AI focused on while detecting missing teeth were visualized using Grad-CAM. Among all the data, the teeth regions were activated. However, differences were observed in the intensity of the red activation depending on the presence of missing teeth and the stage of tooth development. When missing teeth were present, the area surrounding them showed red activation (Fig. 4). In patients without missing teeth, a relatively large area was activated (Fig. 5).

Fig. 4.Grad-CAM overlaid images of the experimental group. The color from blue to red means more activation. Grad-CAM: gradient-weighted class activation mapping.

Fig. 5.Grad-CAM overlaid images of the control group. The color from blue to red means more activation. Grad-CAM: gradient-weighted class activation mapping.

4. Discussion

Panoramic radiographs allow for observation of the entire oral and maxillofacial lesion, while leading to lower radiation exposure than full-mouth periapical radiographs [27]. In pediatric patients, panoramic radiography during regular dental checkups is crucial for identifying various dental anomalies such as supernumerary teeth, tooth agenesis and odontomas. Although panoramic radiographs can only represent three-dimensional structures in a two-dimensional image, recent advances in AI models for automatic interpretation have overcome this challenge [28].

Prior research has demonstrated high accuracy in automatically analyzing the type of implant, location of permanent tooth loss, and remaining alveolar bone height on panoramic radiographs of adults using deep learning models [29, 30, 31]. However, relatively few studies have investigated deep learning models for the automatic analysis of panoramic radiographs in pediatric patients. This may be due to the distinct characteristics of pediatric patients, including the coexistence of primary and permanent teeth with varying stages of eruption and image distortion due to the small face size, making it difficult for AI to learn [7]. To overcome these difficulties, various attempts have been made to remove unnecessary information from panoramic radiographs and set an adequate ROI [10, 11, 32, 33]. According to Dibeh et al. [34], accurate segmentation of the ROI significantly impacts the outcomes of deep learning algorithms. In this study, the ROI was set to a minimal size that included the teeth and surrounding structures, which was sufficient for the deep learning program to learn and analyze the presence of missing teeth. Furthermore, this ROI image encompassed various information, including dental caries and alveolar bone resorption. Therefore, the ROI setting method utilized in this study has the potential to facilitate the automation of additional information analysis in panoramic radiographs.

According to Fujuda et al. [35] and Estai et al. [36], setting a smaller ROI with the necessary information can enhance the accuracy of results. However, this segmentation preprocessing is mostly done manually, and few existing studies describe this process in detail. In this study, the ROI was set by manual labeling, but the process was elucidated in detail. Since panoramic radiographs contain a variety of structures and a vast amount of information, it is inevitable to crop the desired area before the deep learning training. Therefore, the ROI cropping criteria of this study can serve as a basis for setting ROIs for other regions in panoramic radiographs, and further development can contribute to the automation of ROI setting in the future.

According to Hwang et al. [14], previous studies utilizing deep learning used average dataset sizes ranging from 175–1200. Moreover, using extensive data for deep learning training is advantageous [13]. In the data used in this study, the teeth were flipped horizontally and vertically in each quadrant. However, the information necessary for deep learning training and validation remains consistent. Therefore, by dividing one panoramic radiograph into four quadrants, the total dataset size was increased four-fold.

In this study, the average sensitivity and specificity were found to be 0.635 and 0.814, respectively (Table 3). The testing set results of the second cross-validation, which exhibited the highest performance value, were analyzed. First, in cases where the canines were missing, the deep learning model classified them as 100% missing. This outcome could be attributed to the distinct morphological characteristics of canine teeth compared to those of incisors, premolars and molars, which likely favored the deep learning training process. Subsequently, when the upper first premolar, lower first premolar, and lower second premolar were missing, they were classified as missing with probabilities of 84.6%, 80.0% and 77.4%, respectively. According to Heuberer et al. [37], among patients with missing teeth, the premolar area is most frequently missing among the anterior, premolar, and molar areas. In the total data, 46.9% had missing premolars (either the first or second premolar). Therefore, it can be speculated that the high accuracy may have been achieved due to the abundant utilization of radiographs from patients with missing premolars in the training data. Similarly, in the entire dataset, 6.4% exhibited missing molars (either the first or second molar), and the diagnostic accuracy was consistently below 50%. It suggests that the cause of this may be attributed to insufficient training data for missing molars (Fig. 6a). The majority of false positives, accounting for 64.5%, corresponded to permanent dentition. In the Grad-CAM analysis of these errors, activation was primarily observed in the region of the third molar or the inferior region of permanent teeth (Fig. 6b). This suggests that the deep learning model in this study may have attempted to detect the successors even in cases of permanent dentition. Consequently, it is reasonable to conclude that the deep learning model has limitations in its capacity to distinguish primary and permanent teeth, as well as in its ability to classify mixed dentition and permanent dentition.

Fig. 6.Examples of the Grad-CAM of false negative and false positive. (a) Despite the missing of the upper right second molar (#17, black arrow), it was incorrectly diagnosed as being intact. (b) Misdiagnosis due to the recognition of the inferior region of permanent teeth. The color from blue to red means more activation. Grad-CAM: gradient-weighted class activation mapping.

The average AUC of the deep learning model used in this study was 0.730. The AUC represents the classification performance of the deep learning model; thus, this value implies that the deep learning model can correctly classify the presence or absence of teeth with a 73% probability [38]. Comparing these findings with previous results was difficult as the previous study involved a pre-processing step for tooth segmentation prior to evaluating the missing teeth on panoramic images [22]. Different deep learning models have shown AUC values ranging from 0.73 to 0.98 in studies evaluating supernumerary teeth on panoramic radiographs [10, 21]. Thus, the deep learning model in this study showed comparable performance to that of the models used in previous studies. Additionally, Duan et al. [39] reported that an AUC value of 0.9 or higher could be interpreted as an excellent, 0.8–0.9 as a good, 0.7–0.8 as a moderate, and 0.6–0.7 as a mild classification ability. Other studies have used a threshold of 0.7 to assess whether the AUC value indicates sufficient performance and good classification ability for deep learning [40, 41, 42]. Accordingly, the deep learning model in this study can be interpreted as having moderate-to-good distinguishing power.

This study had several limitations. First, panoramic radiographs were divided into four quadrants to evaluate tooth loss. Although this approach increased the data size for deep learning training, it required pre-processing work for labeling the panoramic radiographs. Therefore, the immediate application of this deep learning model in clinical practice is limited. Future research focusing on automating the pre-processing of ROI proposed in this study could overcome this limitation. Second, false positives and false negatives were influenced by differences in the number of the training data for each tooth and variations between mixed and permanent dentition. Therefore, in future research, it may be necessary to balance the training data by accounting for the number of missing teeth. Furthermore, by categorizing data based on the dental age, training the deep learning model could potentially enhance diagnostic accuracy.

5. Conclusions

In this study, a deep learning network model was developed to classify the presence of missing teeth in panoramic radiographs, and its performance was evaluated. The newly defined small-sized ROI proved to be suitable for classifying the presence of missing teeth using this deep learning model. This research may serve as a valuable reference for establishing criteria for automated ROI selection in future panoramic radiograph analysis.

Availability of data and materials

The data presented in this study are available on reasonable request from the corresponding author.

Author contributions

JJH, BHC, EGL and JHS—designed the study. EJK and JJH—performed the research. EJK, JJH and JHS—analyzed the data. EJK—wrote the first draft of the manuscript. JSH—provided critical feedback and approved the final manuscript. All authors contributed to this article. All authors commented on the previous versions of the manuscript.

Ethics approval and consent to participate

This study was approved by the Research Ethics Review Committee (Institutional Review Board, IRB) of Pusan National University Dental Hospital (IRB No.: PNUDH-2022-12-005).

Acknowledgment

Not applicable.

Funding

This research was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI20C0611) and supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI23C0162).

Conflict of interest

The authors declare no conflict of interest.