Automated processing of intraoral clinical photographs using deep learning techniques

1. Introduction

Imaging data play a crucial role in most medical settings. In dentistry, imaging is essential for diagnosing conditions, planning treatment, monitoring progress, and facilitating communication between clinicians and patients [1]. Orthodontists rely on imaging data to make clinical decisions, monitor tooth movement, and plan treatments [1]. Intraoral clinical photographs are non-invasive, radiation-free, and can be archived for long-term storage, thus allowing clinicians to thoroughly examine a patient’s oral condition over time [2]. These images provide detailed information on tooth morphology, alignment, and gingival health. Furthermore, clinical photographs can be used independently to assess clinical characteristics or combined with other diagnostic tools such as plaster models and radiographic images [3].

Digital photographs are an effective diagnostic tool in pediatric dentistry [4, 5, 6, 7, 8]. This advancement supports the relationship between clinicians and patients in terms of diagnosis, clinical documentation, and patient communication [9]. The use of photographs for diagnostic purposes in pediatric dentistry is a cost-effective and non-invasive method [10]. However, capturing high-quality photographs of pediatric patients is challenging because of technical difficulties. Factors such as oral cavity size, patient behavior, and limited clinical time hampers image acquisition, potentially compromising the clinical procedure and image quality. To maintain cooperation, clinicians minimize repeated captures and improve image quality through alignment, cropping, and other adjustments [9].

Traditionally, these images are labeled and stored manually based on their clinical characteristics. However, with advancements in digital dentistry, image data are now more frequently archived in digital clinical record systems, streamlining access for future evaluation and patient follow-up [11]. Efficient use of image data requires basic categorization and enhancements in quality and consistency through editing [12]. Therefore, developing a system integrating automated classification and editing would be beneficial for improving dental practice efficiency and reducing specialist workload.

As artificial intelligence (AI) becomes a new standard in dentistry and healthcare, deep learning, a specialized branch of AI that utilizes convolutional neural networks (CNNs), is now widely applied to the analysis of image data [3]. Currently, deep learning is used in dental image processing, demonstrating high accuracy in detecting plaque and caries, assessing the need for extraction-based orthodontics, and automatically classifying images [1, 2, 3, 13, 14].

Previous studies using deep learning for intraoral image classification have reported an accuracy of greater than 98% in distinguishing intraoral and extraoral clinical photographs [1, 3]. In addition, most commercially available deep learning-based diagnostic software systems—such as CellmatiQ (DentaliQ ortho, CellmatiQ GmbH, Hamburg, HH, Germany), ORCA AI (CephX v4.02, ORCA Dental AI Ltd., Herzliya, Israel), and WebCeph (version 1.0.0, AssembleCircle Corp., Gyeonggi-do, Korea)—primarily support tasks such as landmark detection and cephalometric analysis [15]. While some CNN-based models have been developed for classifying views or identifying dental findings and anatomical regions [3, 13], these approaches have primarily focused on single-purpose tasks. Moreover, to the best of our knowledge, additional preprocessing tasks, such as image orientation and cropping, require new deep learning approaches that have been rarely addressed in previous studies or integrated into existing systems [16].

Accordingly, this study implemented two CNNs, each specialized in image editing and classification. We aim to enhance the general efficiency and precision of the processing workflow by assigning each network a specific function [17]. This study focused on building a deep learning framework that can classify and edit five types of intraoral clinical photographs (frontal, left buccal, right buccal, upper occlusal, and lower occlusal) obtained from pediatric and adolescent patients.

2. Materials and methods

2.4 Training and five-fold cross-validation

All models were trained using MATLAB 2024b equipped with Deep Learning and Parallel Computing Toolboxes running on an NVIDIA RTX 4090 GPU (Santa Clara, CA, USA) (24 GB Random Access Memory (RAM)). The intraoral photo orientation correction network based on Inception-ResNet-v2 (Fig. 1a) was optimized using the Adam optimizer with a fixed initial learning rate of 1 × 10⁻⁴. Training was conducted for a maximum of 100 epochs, using a minibatch size of 16. Validation was performed every 20 iterations, and early stopping was applied with a patience of 30 validation checks. The region of interest detection network based on Faster R-CNN (Fig. 1b) also used the Adam optimizer but employed a piecewise learning rate schedule, where the learning rate was reduced by a factor of 0.2 every 10 epochs. This network was likewise trained for up to 100 epochs with a minibatch size of 16, using the same validation strategy and early stopping criteria. In both cases, the training data were shuffled at the beginning of each epoch, and five-fold cross-validation was used to ensure robust performance evaluation.

Fig. 1.Schematic of the dual deep learning architecture. (a) Intraoral photo orientation correction network based on Inception-ResNet-v2. (b) ROI detection network based on Faster R-CNN. MaxPool: Max pooling; AvgPool: Average pooling; Concat: Concatenation; ROI: Region of interest; RPN: Region proposal network; Bbox_pred: Bounding box prediction; Cls-prob: Class probability; Im_info: Image information.

For training and validation, all the images were resized to a format of 299 × 299 pixels to meet the input size requirements of the Inception-ResNet-v2 architecture, ensuring compatibility with the network and standardized preprocessing. To ensure stable convergence across cross-validation folds, the target labels (rotation angles) were standardized using z-score normalization, which centers the data around a mean of zero with unit variance. This approach was chosen to normalize the continuous angle values, thereby improving numerical stability during backpropagation and facilitating efficient optimization in the regression network. The parameters (mean and standard deviation) used for normalization were calculated from the combined training and validation sets, and the predicted outputs were denormalized back to degree values for evaluation and interpretation. Data augmentation was applied during training using the imageDataAugmenter function of MATLAB. The augmentation pipeline included random rotations (±5°), scaling (0.8–1.2×), and horizontal/vertical translations (±10 pixels), which were applied on-the-fly to introduce variability while preserving anatomical integrity. A five-fold cross-validation approach was adopted to improve generalizability and mitigate data bias considering the limited sample size. The dataset was randomly partitioned into five subsets, of which four subsets were allocated for training and one for validation in each iteration. This procedure was iterated five times, with each subset being used once as the validation set, thereby ensuring efficient and comprehensive utilization of the entire dataset.

2.5 Performance metrics

To assess the regression performance for intraoral photo orientation correction, we employed two standard metrics: root mean squared error (RMSE) and mean absolute error (MAE). These are defined in Eqns. 1,2, where y, ŷ, and ȳ denote the ground truth, predicted value, and mean of the actual values, respectively, and n denotes the number of samples.

$RMSE=\sqrt{\frac{\sum _{i=1}^n{(y_i-\widehat{y_i})}^2}{n}}$ (1)

$MAE=\frac{1}{n}\sum _{i=1}^n|y_i-{\hat{y}}_i|$ (2)

For classification performance, the accuracy was computed by comparing the predicted and actual labels for each detected region. The label with the highest confidence score is considered the final prediction, as shown in Eqn. 3. Additionally, a confusion matrix and receiver operating characteristic (ROC) curve were evaluated to assess classification performance.

$ClassificatonAccuracy=\frac{NumberofCorrectPredictions}{TotalNumberofPredictions}$ (3)

The IoU metric, the ratio of the overlapping area to the union of the predicted and ground truth bounding boxes (Eqn. 4), was used to evaluate the localization performance.

$IoU=\frac{AreaofOverlap}{AreaofUnion}$ (4)

The precision and recall were calculated using true positives (TP), false positives (FP), and false negatives (FN), with IoU thresholds set at 0.5, 0.75, and 0.5, respectively 0.95 in 0.05 increments (Eqns. 5,6).

$Precision=\frac{TP}{TP+FP}$ (5)

$Recall=\frac{TP}{TP+FN}$ (6)

Finally, the average precision (AP), a widely adopted metric in object detection, was computed as the area under the precision-recall (PR) curve (Eqn. 7).

$AP={\int }_0^{1}P\left(r\right)dr$ (7)

The overall workflow is illustrated in Fig. 2.

Fig. 2.Schematic of the workflow. Original intraoral images were rotated and labeled. A regression network (Inception-ResNet-v2) was used to correct image orientation, and a detection network (Faster R-CNN) was used to classify image types. A 5-fold cross-validation was conducted to train and evaluate the deep learning model using 3000 images. CNN: convolutional neural network.

3. Results

Both the intraoral photo orientation correction network based on Inception-ResNet-v2 and the region of interest detection network employing Faster R-CNN were evaluated through a five-fold cross-validation procedure to ensure robust performance assessment. The training curves for both networks exhibited consistent and gradual convergence of training and validation losses across all folds (Fig. 3). Although minor differences between training and validation losses were observed—reflecting typical generalization dynamics—the validation losses steadily approached values close to those of the training losses. This behavior indicates effective learning with minimal overfitting throughout the cross-validation process.

Fig. 3.Training and validation loss curves for the two deep learning models evaluated using five-fold cross-validation. (a) Inception-ResNet-v2. (b) Faster R-CNN.

3.2 Region of interest detection network

The outcomes of five-fold cross-validation, including the classification accuracy and AP scores under various IoU thresholds, are listed in Table 2. The mean classification accuracy for the frontal, left buccal, right buccal, upper occlusal, and lower occlusal images were 0.997, 0.977, 0.995, 0.993 and 0.988, respectively. The model’s performance was evaluated using a confusion matrix and demonstrated excellent results in the classification of clinical images (Fig. 4a). Additionally, an ROC Curve was employed to assess the model’s performance across different thresholds, and the high Area Under the Curve (AUC) further confirmed the model’s robust overall performance (Fig. 4b).

Table 2.Performance of classification and region of interest detection using 5-fold cross-validation.

Fold	Regions	Classification accuracy	AP (IoU threshold 0.5)	AP (IoU threshold 0.75)	AP (IoU threshold 0.5–0.95)
1
	Frontal	0.992	0.911	0.820	0.609
	Left buccal	0.975	0.938	0.641	0.421
	Right buccal	0.992	0.956	0.696	0.473
	Upper	0.992	0.980	0.798	0.661
	Lower	0.983	0.942	0.872	0.707
2
	Frontal	0.992	0.956	0.823	0.443
	Left buccal	0.983	0.946	0.648	0.425
	Right buccal	0.992	0.945	0.744	0.578
	Upper	0.992	0.988	0.863	0.672
	Lower	0.983	0.965	0.886	0.777
3
	Frontal	1.000	0.941	0.841	0.616
	Left buccal	0.975	0.961	0.651	0.440
	Right buccal	0.992	0.954	0.705	0.472
	Upper	0.992	0.948	0.782	0.635
	Lower	0.983	0.937	0.861	0.684
4
	Frontal	1.000	0.964	0.795	0.447
	Left buccal	0.975	0.960	0.702	0.480
	Right buccal	1.000	0.957	0.770	0.620
	Upper	0.992	0.966	0.869	0.666
	Lower	0.992	0.971	0.856	0.772
5
	Frontal	1.000	0.960	0.817	0.467
	Left buccal	0.975	0.954	0.680	0.460
	Right buccal	1.000	0.952	0.775	0.609
	Upper	1.000	0.962	0.855	0.690
	Lower	1.000	0.969	0.898	0.801
Average
	Frontal	0.997	0.946	0.819	0.517
	Left buccal	0.977	0.952	0.664	0.445
	Right buccal	0.995	0.962	0.738	0.550
	Upper	0.993	0.969	0.833	0.665
	Lower	0.988	0.965	0.875	0.748
AP: average precision; IoU: Intersection over Union.

Fig. 4.Classification performance of the Inception-ResNet-v2 model for intraoral image classification. (a) Confusion matrix. (b) ROC curve. ROC: Receiver operating characteristic.

To compute the AP values, PR curves were derived and differed depending on the applied IoU threshold. For our validation dataset, PR curves were generated using IoU thresholds of 0.5, 0.75, and a range spanning from 0.5 to 0.95 in increments of 0.05. As the threshold increased from 0.5 to 0.75 and then to 0.95, a decline in model performance was observed.

AP scores were calculated individually for all five intraoral regions. At the IoU threshold of 0.5, the regions with the highest to lowest AP were the upper occlusal, lower occlusal, right buccal, left buccal, and frontal regions. At a 0.75 threshold, the order changed to lower occlusal, upper occlusal, frontal, right buccal, and left buccal. When evaluated over the full IoU range (0.5–0.95 with 0.05 increments), lower occlusal again yielded the highest average AP, followed by upper occlusal, right buccal, frontal, and left buccal, respectively (Fig. 5).

Fig. 5.Variation in precision–recall (PR) curves across different intersection over union (IoU) thresholds. (a) Frontal. (b) Left buccal. (c) Right buccal. (d) Upper occlusal. (e) Lower occlusal. PR curves were generated at IoU thresholds of 0.50 (blue), 0.75 (orange), and 0.50–0.95 in 0.05 increments (yellow).

Fig. 6 shows a sample input image fed into the CNN model, model output, and the final cropped region of interest.

Fig. 6.Example output showing orientation correction and region classification by the model. (a) The blue bounding box indicates the ground truth region of interest, annotated using the Image Labeler in MATLAB 2024b (MathWorks Inc., Natick, MA, USA), while yellow bounding box represents the model-predicted region of interest. (b) Representative failure cases demonstrating the model’s limitations. From left to right, the images show a poorly localized region, an image with fogging on the intraoral mirror, and soft tissue interference caused by a retractor.

4. Discussion

In this study, we developed a fully automated system for classifying and editing intraoral clinical photographs, and the evaluation results demonstrated high accuracy.

The Inception-ResNet-v2-based regression model used in this study achieved a remarkably low orientation error, with a mean RMSE of 0.571° and an MAE of 0.407°. Although both metrics quantify the discrepancy between predicted and actual orientation angles, they are complementary: RMSE penalizes larger errors more heavily and is thus sensitive to outliers, while MAE provides a direct and more interpretable measure of the average error magnitude [23, 24]. These values are clinically significant because RMSE and MAE fall well below the perceptual thresholds for angular detection reported in the literature—approximately 2° for dental professionals and 4° for laypersons [25]. This sub-degree level of precision suggests that the orientation correction achieved by our model is unlikely to be perceived by clinicians, thereby reinforcing its practical utility in the standardization of intraoral photographs and the automation of clinical workflows.

Notably, we developed an automatic orientation alignment model that considers various tooth eruption patterns in pediatric patients [26]. Although previous studies have attempted to identify the regions of interest in intraoral photographs [1, 3], research focusing exclusively on pediatric patients is scarce. Furthermore, to the best of our knowledge, no study has specifically addressed image-orientation alignment in pediatric intraoral photography. The proposed model was designed to achieve consistent rotational alignment even in pediatric patients with diverse tooth eruption stages and limited behavioral cooperation. By replacing the repetitive and time-consuming task of manual orientation alignment, the proposed model is expected to contribute to the standardization of image analysis and improve clinical efficiency in pediatric dental practice.

The region of interest detection model based on Faster R-CNN demonstrated high performance, achieving a mean AP of over 0.946 at an IoU threshold of 0.5. However, the detection performance exhibited a declining trend as the IoU threshold increased (AP = 0.75, AP = 0.5–0.95). This decrease may reflect the inherent difficulty of identifying precise locations in clinical intraoral photographs, particularly in regions with complex soft tissue boundaries or variable imaging conditions, as higher IoU thresholds demand pixel-level accuracy. Among the various views, the highest detection performance was observed in the upper and lower occlusal surfaces, whereas a relatively lower performance was observed in the left and right buccal views. The superior performance of occlusal views may be attributed to their relatively consistent anatomical structures [27, 28] and the lower likelihood of interference from soft tissues during image acquisition, such as the tongue or buccal mucosa, which allows for more consistent imaging. However, buccal images are more prone to occlusion by intraoral structures and are more sensitive to variations in image quality due to differing capture conditions [28]. These factors may hinder the model’s capacity to learn consistent features, which in turn may reduce detection performance. Therefore, further research is needed to improve the detection performance in buccal views. We hypothesize that expanding the dataset with additional intraoral images would further improve model performance. In addition, strategies such as standardizing image acquisition protocols and incorporating additional angled buccal views may enhance feature consistency and improve detection accuracy in future models [29].

When comparing the classification accuracy of intraoral clinical photographs with those of previous studies [1, 3], the present study achieved an accuracy of 99.0%, whereas Ryu et al. [3] and Li et al. [1] reported accuracies of 98.0% and 99.4%, respectively. Although slight differences in performance were observed due to variations in the model architecture and validation methods, the classification performance in this study was similarly high, which is consistent with previous findings. This high accuracy may be partially explained by the use of cross validation, which provides more stability and, in some cases, higher performance estimates than single-train–test split validation, particularly in studies with limited sample sizes [30].

These findings have important clinical implications. Compared with manual image processing, which is time-consuming and prone to interoperator variability [31], this level of automation enables a more efficient and consistent workflow. AI-based approaches can match or even surpass clinician performance in various diagnostic imaging tasks [11]. In addition, such systems have been shown to reduce the burden of repetitive and labor-intensive procedures in clinical dental practice [11, 12, 31]. Building on these findings, the present study represents a step toward the integration of deep learning tools into pediatric dental care, contributing to improved standardization, reliability, and timeliness in image-based assessments.

When compared directly with human evaluators using the test set, the deep-learning (DL) model demonstrated a substantial reduction in evaluation time (0.11 seconds per image) relative to dentists (20.67 ± 4.95 seconds), emphasizing its potential value in time-sensitive clinical workflows. However, it should be noted that the model exhibited slightly lower accuracy in orientation correction and region detection than human evaluators. This discrepancy may indicate a degree of overfitting or sample bias, particularly given the relatively small dataset. As with most deep learning approaches, our model is subject to general limitations such as overfitting, data-specific bias, and limited interpretability. These challenges underscore the importance of integrating explainable AI techniques and validating model performance across external datasets [32].

This study had several limitations. First, all photographs were obtained from a single institution. Although they were captured by multiple dentists using various cameras and acquisition settings, this internal variability may not sufficiently mitigate potential biases arising from institution-specific imaging protocols and patient populations. Second, despite implementing data augmentation and five-fold cross-validation to enhance model performance and mitigate overfitting, the relatively limited dataset size remains a significant limitation. To improve the robustness and generalizability of the proposed model, future studies should incorporate larger datasets, including external or multicenter validation. In particular, incorporating more demographically, clinically, and technically diverse data could broaden the model’s applicability beyond pediatric orthodontics to adult populations and other dental specialties, thereby enhancing its clinical utility. Third, the retrospective nature of this study limited our ability to perform a stratified analysis of model performance based on device type or image acquisition parameters, as metadata regarding camera models, specific settings, and operator identity were unavailable. Variability in medical image data—arising from differences in clinical protocols, imaging devices, acquisition settings (such as resolution, lighting, and focus), and operator technique—can lead to inconsistencies in image quality. These inconsistencies can significantly affect the performance and generalizability of deep learning models [33, 34, 35]. In particular, low-quality images—such as those affected by blur, low resolution, or noise—have been shown to reduce the accuracy of CNN-based models, especially when such cases are underrepresented in the training dataset [36]. Therefore, future studies should prospectively collect detailed acquisition metadata and incorporate objective image quality metrics to enable subgroup analyses. Moreover, incorporating a wider range of image qualities—including suboptimal images commonly encountered in clinical practice—and applying data augmentation techniques such as color and contrast adjustments, as well as degradations like noise and blur, may collectively enhance the model’s robustness to real-world variability [37].

In addition to addressing these limitations, future studies evaluating clinician acceptance and workflow integration will be essential to ensure the successful use of AI-based tools in pediatric dentistry. To maximize clinical utility, future research could also examine the performance of the model across diverse patient populations and imaging conditions. The dual deep learning model adopted in this study also holds promise for future research. Particularly, the classified and edited intraoral clinical photographs may be further used for simultaneous tasks, such as identifying restoration types and detecting dental caries. Moreover, the system could be integrated into broader diagnostic workflows, such as orthodontic assessments and patient education tools.

5. Conclusions

In this study, we developed a deep learning-based model for the automated processing of five standard types of intraoral clinical photographs—including orientation correction, cropping, and classification—in pediatric dental patients. The model yielded highly accurate regression and detection results, indicating its potential to reduce inconsistencies and repetitive tasks among clinicians during image processing. By incorporating images from pediatric and adolescent patients under various clinical conditions, the model demonstrates strong potential for integration into pediatric dental workflows. It may serve as a valuable tool in pediatric dentistry by streamlining image processing and supporting standardized diagnosis and documentation in clinical settings.

Abbreviations

AI, Artificial intelligence; AP, Average precision; CNN, Convolutional neural networks; FN, False negatives; FP, False positives; IoU, Intersection over Union; MAE, Mean absolute error; PR, Precision-recall; RMSE, Root mean squared error; TP, True positives; DSLR, digital single-lens reflex; ICC, intraclass correlation coefficient; MaxPool, Max pooling; AvgPool, Average pooling; Concat, Concatenation; ROI, Region of interest; RPN, Region proposal network; Bbox_pred, Bounding box prediction; Cls_prob, Class probability; Im_info, Image information; ROC, Receiver operating characteristic; AUC, Area Under the Curve; DL, deep-learning; R-CNN, Region-based Convolutional Neural Network; ISO, International Organization for Standardization; JPEG, Joint Photographic Experts Group; ID, Identifier; RAM, Random Access Memory.

Availability of data and materials

The data presented in this study are available on reasonable request from the corresponding author.

Author contributions

HS, TJ, EL and JS—designed the study. JE and HS—performed the research. JE, HS and SP—analyzed the data. JE—wrote the first draft of the manuscript. JS—provided critical feedback and approved the final manuscript. All authors contributed to this article. All authors commented on the previous versions of the manuscript.

Ethics approval and consent to participate

This study was approved by the Institutional Review Boards (IRB) of the Pusan National University Dental Hospital (IRB approval number: PNUDH 2025-02-010). Given the retrospective nature of the study, the Institutional Review Board of the Pusan National University Dental Hospital waived the requirement for written informed consent.

Acknowledgment

Not applicable.

Funding

This research received no external funding.

Conflict of interest

The authors declare no conflict of interest.