Commercial artificial intelligence lateral cephalometric analysis: part 1—the possibility of replacing manual landmarking with artificial intelligence service

Artificial intelligence (AI) technology has recently been introduced to dentistry. AI-assisted cephalometric analysis is one of its applications, and several commercial AI services have already been launched. However, the performance of these commercial services is still unclear. This study aimed to determine whether commercially available AI cephalometric analysis can replace manual analysis by human examiners. Eighty-four pretreatment lateral cephalograms were traced and examined by two orthodontists and four commercial AIs, and 13 commonly used cephalometric variables were calculated. Then, the Bland-Altman analysis was conducted to evaluate systematic and random errors between examiners. The interchangeability of an AI was determined if the random errors of the AI were smaller than the clinically acceptable limits derived from the random errors between human examiners. Finally, the inter-examiner reliability index was calculated, and Cohen’s kappa was determined to assess the actual classification reliability of each examiner. The systematic errors of the AIs were clinically insignificant in general. However, the random errors of the AIs were approximately twice those of human examiners, which did not satisfy the interchangeability condition. Furthermore, even though the reliability indices of the AIs were in the good-to-excellent range, their classification reliability was unacceptable. Commercial AI is still at a level that makes it challenging to replace manual landmarking by human experts. Thus, a human examiner’s landmark position review is mandatory when using commercial AIs.

Cephalometric; Artificial intelligence; Accuracy; Precision; Reliability

[1] Choi YJ, Lee K. Possibilities of artificial intelligence use in orthodontic diagnosis and treatment planning: image recognition and three-dimensional VTO. Seminars in Orthodontics. 2021; 27: 121–129.

[2] Kim H, Shim E, Park J, Kim YJ, Lee U, Kim Y. Web-based fully automated cephalometric analysis by deep learning. Computer Methods and Programs in Biomedicine. 2020; 194: 105513.

[3] Hwang HW, Moon JH, Kim MG, Donatelli RE, Lee SJ. Evaluation of automated cephalometric analysis based on the latest deep learning method. The Angle Orthodontist. 2021; 91: 329–335.

[4] Gil SM, Kim I, Cho JH, Hong M, Kim M, Kim SJ, et al. Accuracy of auto-identification of the posteroanterior cephalometric landmarks using cascade convolution neural network algorithm and cephalometric images of different quality from nationwide multiple centers. American Journal of Orthodontics and Dentofacial Orthopedics. 2022; 161: e361–e371.

[5] Yoon HJ, Kim DR, Gwon E, Kim N, Baek SH, Ahn HW, et al. Fully automated identification of cephalometric landmarks for upper airway assessment using cascaded convolutional neural networks. European Journal of Orthodontics. 2022; 44: 66–77.

[6] Park JH, Hwang HW, Moon JH, Yu Y, Kim H, Her SB, et al. Automated identification of cephalometric landmarks: part 1—comparisons between the latest deep-learning methods YOLOV3 and SSD. The Angle Orthodontist. 2019; 89: 903–909.

[7] Hwang HW, Park JH, Moon JH, Yu Y, Kim H, Her SB, et al. Automated identification of cephalometric landmarks: part 2—might it be better than human? The Angle Orthodontist. 2020; 90: 69–76.

[8] Lagravère MO, Low C, Flores-Mir C, Chung R, Carey JP, Heo G, et al. Intraexaminer and interexaminer reliabilities of landmark identification on digitized lateral cephalograms and formatted 3-dimensional cone-beam computerized tomography images. American Journal of Orthodontics and Dentofacial Orthopedics. 2010; 137: 598–604.

[9] Tanikawa C, Oka A, Lim J, Lee C, Yamashiro T. Clinical applicability of automated cephalometric landmark identification: Part II—number of images needed to re‐learn various quality of images. Orthodontics & Craniofacial Research. 2021; 24: 53–58.

[10] Bulatova G, Kusnoto B, Grace V, Tsay TP, Avenetti DM, Sanchez FJC. Assessment of automatic cephalometric landmark identification using artificial intelligence. Orthodontics & Craniofacial Research. 2021; 24: 37–42.

[11] Schwendicke F, Chaurasia A, Arsiwala L, Lee J, Elhennawy K, Jost-Brinkmann PG, et al. Deep learning for cephalometric landmark detection: systematic review and meta-analysis. Clinical Oral Investigations. 2021; 25: 4299–4309.

[12] Jeon S, Lee KC. Comparison of cephalometric measurements between conventional and automatic cephalometric analysis using convolutional neural network. Progress in Orthodontics. 2021; 22: 14.

[13] Kunz F, Stellzig-Eisenhauer A, Zeman F, Boldt J. Artificial intelligence in orthodontics: evaluation of a fully automated cephalometric analysis using a customized convolutional neural network. Journal of Orofacial Orthopedics. 2020; 81: 52–68.

[14] Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine. 2016; 15: 155–163.

[15] Haghayegh S, Kang HA, Khoshnevis S, Smolensky MH, Diller KR. A comprehensive guideline for Bland-Altman and intra class correlation calculations to properly compare two methods of measurement and interpret findings. Physiological Measurement. 2020; 41: 055012.

[16] Bland JM, Altman DG. Measuring agreement in method comparison studies. Statistical Methods in Medical Research. 1999; 8: 135–160.

[17] van Stralen KJ, Jager KJ, Zoccali C, Dekker FW. Agreement between methods. Kidney International. 2008; 74: 1116–1120.

[18] Tanikawa C, Lee C, Lim J, Oka A, Yamashiro T. Clinical applicability of automated cephalometric landmark identification: part I—patient-related identification errors. Orthodontics & Craniofacial Research. 2021; 24: 43–52.

[19] Rudolph DJ, Sinclair PM, Coggins JM. Automatic computerized radiographic identification of cephalometric landmarks. American Journal of Orthodontics and Dentofacial Orthopedics. 1998; 113: 173–179.

[20] Kazandjian S, Kiliaridis S, Mavropoulos A. Validity and reliability of a new edge-based computerized method for identification of cephalometric landmarks. The Angle Orthodontist. 2006; 76: 619–624.