Commercial artificial intelligence lateral cephalometric analysis: part 1—the possibility of replacing manual landmarking with artificial intelligence service
1Department of Pediatric Dentistry, School of Dentistry, Kyungpook National University, 41940 Daegu, Republic of Korea
2Department of Orthodontics, School of Dentistry, Kyungpook National University, 41940 Daegu, Republic of Korea
DOI: 10.22514/jocpd.2023.085 Vol.47,Issue 6,November 2023 pp.106-118
Submitted: 10 April 2023 Accepted: 12 May 2023
Published: 03 November 2023
† These authors contributed equally.
Artificial intelligence (AI) technology has recently been introduced to dentistry. AI-assisted cephalometric analysis is one of its applications, and several commercial AI services have already been launched. However, the performance of these commercial services is still unclear. This study aimed to determine whether commercially available AI cephalometric analysis can replace manual analysis by human examiners. Eighty-four pretreatment lateral cephalograms were traced and examined by two orthodontists and four commercial AIs, and 13 commonly used cephalometric variables were calculated. Then, the Bland-Altman analysis was conducted to evaluate systematic and random errors between examiners. The interchangeability of an AI was determined if the random errors of the AI were smaller than the clinically acceptable limits derived from the random errors between human examiners. Finally, the inter-examiner reliability index was calculated, and Cohen’s kappa was determined to assess the actual classification reliability of each examiner. The systematic errors of the AIs were clinically insignificant in general. However, the random errors of the AIs were approximately twice those of human examiners, which did not satisfy the interchangeability condition. Furthermore, even though the reliability indices of the AIs were in the good-to-excellent range, their classification reliability was unacceptable. Commercial AI is still at a level that makes it challenging to replace manual landmarking by human experts. Thus, a human examiner’s landmark position review is mandatory when using commercial AIs.
Cephalometric; Artificial intelligence; Accuracy; Precision; Reliability
Jaesik Lee,Seong-Ryeol Bae,Hyung-Kyu Noh. Commercial artificial intelligence lateral cephalometric analysis: part 1—the possibility of replacing manual landmarking with artificial intelligence service. Journal of Clinical Pediatric Dentistry. 2023. 47(6);106-118.
 Choi YJ, Lee K. Possibilities of artificial intelligence use in orthodontic diagnosis and treatment planning: image recognition and three-dimensional VTO. Seminars in Orthodontics. 2021; 27: 121–129.
 Kim H, Shim E, Park J, Kim YJ, Lee U, Kim Y. Web-based fully automated cephalometric analysis by deep learning. Computer Methods and Programs in Biomedicine. 2020; 194: 105513.
 Hwang HW, Moon JH, Kim MG, Donatelli RE, Lee SJ. Evaluation of automated cephalometric analysis based on the latest deep learning method. The Angle Orthodontist. 2021; 91: 329–335.
 Gil SM, Kim I, Cho JH, Hong M, Kim M, Kim SJ, et al. Accuracy of auto-identification of the posteroanterior cephalometric landmarks using cascade convolution neural network algorithm and cephalometric images of different quality from nationwide multiple centers. American Journal of Orthodontics and Dentofacial Orthopedics. 2022; 161: e361–e371.
 Yoon HJ, Kim DR, Gwon E, Kim N, Baek SH, Ahn HW, et al. Fully automated identification of cephalometric landmarks for upper airway assessment using cascaded convolutional neural networks. European Journal of Orthodontics. 2022; 44: 66–77.
 Park JH, Hwang HW, Moon JH, Yu Y, Kim H, Her SB, et al. Automated identification of cephalometric landmarks: part 1—comparisons between the latest deep-learning methods YOLOV3 and SSD. The Angle Orthodontist. 2019; 89: 903–909.
 Hwang HW, Park JH, Moon JH, Yu Y, Kim H, Her SB, et al. Automated identification of cephalometric landmarks: part 2—might it be better than human? The Angle Orthodontist. 2020; 90: 69–76.
 Lagravère MO, Low C, Flores-Mir C, Chung R, Carey JP, Heo G, et al. Intraexaminer and interexaminer reliabilities of landmark identification on digitized lateral cephalograms and formatted 3-dimensional cone-beam computerized tomography images. American Journal of Orthodontics and Dentofacial Orthopedics. 2010; 137: 598–604.
 Tanikawa C, Oka A, Lim J, Lee C, Yamashiro T. Clinical applicability of automated cephalometric landmark identification: Part II—number of images needed to re‐learn various quality of images. Orthodontics & Craniofacial Research. 2021; 24: 53–58.
 Bulatova G, Kusnoto B, Grace V, Tsay TP, Avenetti DM, Sanchez FJC. Assessment of automatic cephalometric landmark identification using artificial intelligence. Orthodontics & Craniofacial Research. 2021; 24: 37–42.
 Schwendicke F, Chaurasia A, Arsiwala L, Lee J, Elhennawy K, Jost-Brinkmann PG, et al. Deep learning for cephalometric landmark detection: systematic review and meta-analysis. Clinical Oral Investigations. 2021; 25: 4299–4309.
 Jeon S, Lee KC. Comparison of cephalometric measurements between conventional and automatic cephalometric analysis using convolutional neural network. Progress in Orthodontics. 2021; 22: 14.
 Kunz F, Stellzig-Eisenhauer A, Zeman F, Boldt J. Artificial intelligence in orthodontics: evaluation of a fully automated cephalometric analysis using a customized convolutional neural network. Journal of Orofacial Orthopedics. 2020; 81: 52–68.
 Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine. 2016; 15: 155–163.
 Haghayegh S, Kang HA, Khoshnevis S, Smolensky MH, Diller KR. A comprehensive guideline for Bland-Altman and intra class correlation calculations to properly compare two methods of measurement and interpret findings. Physiological Measurement. 2020; 41: 055012.
 Bland JM, Altman DG. Measuring agreement in method comparison studies. Statistical Methods in Medical Research. 1999; 8: 135–160.
 van Stralen KJ, Jager KJ, Zoccali C, Dekker FW. Agreement between methods. Kidney International. 2008; 74: 1116–1120.
 Tanikawa C, Lee C, Lim J, Oka A, Yamashiro T. Clinical applicability of automated cephalometric landmark identification: part I—patient-related identification errors. Orthodontics & Craniofacial Research. 2021; 24: 43–52.
 Rudolph DJ, Sinclair PM, Coggins JM. Automatic computerized radiographic identification of cephalometric landmarks. American Journal of Orthodontics and Dentofacial Orthopedics. 1998; 113: 173–179.
 Kazandjian S, Kiliaridis S, Mavropoulos A. Validity and reliability of a new edge-based computerized method for identification of cephalometric landmarks. The Angle Orthodontist. 2006; 76: 619–624.
Science Citation Index Expanded (SciSearch) Created as SCI in 1964, Science Citation Index Expanded now indexes over 9,500 of the world’s most impactful journals across 178 scientific disciplines. More than 53 million records and 1.18 billion cited references date back from 1900 to present.
PubMed (MEDLINE) PubMed comprises more than 35 million citations for biomedical literature from MEDLINE, life science journals, and online books. Citations may include links to full text content from PubMed Central and publisher web sites.
Biological Abstracts Easily discover critical journal coverage of the life sciences with Biological Abstracts, produced by the Web of Science Group, with topics ranging from botany to microbiology to pharmacology. Including BIOSIS indexing and MeSH terms, specialized indexing in Biological Abstracts helps you to discover more accurate, context-sensitive results.
Google Scholar Google Scholar is a freely accessible web search engine that indexes the full text or metadata of scholarly literature across an array of publishing formats and disciplines.
JournalSeek Genamics JournalSeek is the largest completely categorized database of freely available journal information available on the internet. The database presently contains 39226 titles. Journal information includes the description (aims and scope), journal abbreviation, journal homepage link, subject category and ISSN.
Current Contents - Clinical Medicine Current Contents - Clinical Medicine provides easy access to complete tables of contents, abstracts, bibliographic information and all other significant items in recently published issues from over 1,000 leading journals in clinical medicine.
BIOSIS Previews BIOSIS Previews is an English-language, bibliographic database service, with abstracts and citation indexing. It is part of Clarivate Analytics Web of Science suite. BIOSIS Previews indexes data from 1926 to the present.
Journal Citation Reports/Science Edition Journal Citation Reports/Science Edition aims to evaluate a journal’s value from multiple perspectives including the journal impact factor, descriptive data about a journal’s open access content as well as contributing authors, and provide readers a transparent and publisher-neutral data & statistics information about the journal.
Scopus: CiteScore 2.0 (2022) Scopus is Elsevier's abstract and citation database launched in 2004. Scopus covers nearly 36,377 titles (22,794 active titles and 13,583 Inactive titles) from approximately 11,678 publishers, of which 34,346 are peer-reviewed journals in top-level subject fields: life sciences, social sciences, physical sciences and health sciences.