Transfer Deep Learning for Dental and Maxillofacial Imaging Modality Classification: A Preliminary Study

Objective: To apply the technique of transfer deep learning on a small data set for automatic classification of X-ray modalities in dentistry. Study design: For solving the problem of classification, the convolution neural networks based on VGG16, NASNetLarge and Xception architectures were used, which received pre-training on ImageNet subset. In this research, we used an in-house dataset created within the School of Dental Medicine, Tel Aviv University. The training dataset contained anonymized 496 digital Panoramic and Cephalometric X-ray images for orthodontic examinations from CS 8100 Digital Panoramic System (Carestream Dental LLC, Atlanta, USA). The models were trained using NVIDIA GeForce GTX 1080 Ti GPU. The study was approved by the ethical committee of Tel Aviv University. Results: The test dataset contained 124 X-ray images from 2 different devices: CS 8100 Digital Panoramic System and Planmeca ProMax 2D (Planmeca, Helsinki, Finland). X-ray images in the test database were not pre-processed. The accuracy of all neural network architectures was 100%. Following a result of almost absolute accuracy, the other statistical metrics were not relevant. Conclusions: In this study, good results have been obtained for the automatic classification of different modalities of X-ray images used in dentistry. The most promising direction for the development of this kind of application is the transfer deep learning. Further studies on automatic classification of modalities, as well as sub-modalities, can maximally reduce occasional difficulties arising in this field in the daily practice of the dentist and, eventually, improve the quality of diagnosis and treatment.

Neural network; Deep learning; Classification; Dental imaging modality; Maxillofacial imaging modality; Classification of X ray modalities

1. Yu Yuhai, Lin Hongfei, Yu Q, Meng J, Zhao Z, Li, Yanpeng, Zuo L. Modality classification for medical images using multiple deep convolutional neural networks. Journal of Computational Information Systems 11: 5403-13, 2015.

2. Hamet P, Tremblay J. Artificial intelligence in medicine. Metabolism 69: 36- 40, 2017.

3. Hosny A, Parmar C, Quackenbush J, Schwartz LH, Aerts HJWL. Artificial intelligence in radiology. Nat Rev Cancer 18: 500-10, 2018.

4. Yasaka K, Akai H, Kunimatsu A, Abe O, Kiryu S. Liver Fibrosis: Deep Convolutional Neural Network for Staging by Using Gadoxetic Acid-enhanced Hepatobiliary Phase MR Images. Radiology 287: 146–55, 2018.

5. van Ginneken B. Fifty years of computer analysis in chest imaging: rule-based, machine learning, deep learning. Radiol Phys Technol 10: 23-32, 2017.

6. Avanzo M, Stancanello J, Pirrone G, Sartor G. Radiomics and deep learni ng in lung cancer. Strahlenther Onkol 196: 879-87, 2020.

7. Kooi T, Litjens G, van Ginneken B, Gubern-Mérida A, Sánchez CI, Mann R, den Heeten A, Karssemeijer N. Large scale deep learning for computer aided detection of mammographic lesions. Med Image Anal 35: 303-12, 2017.

8. Charron O, Lallement A, Jarnet D, Noblet V, Clavier JB, Meyer P. Automatic detection and segmentation of brain metastases on multimodal MR images with a deep convolutional neural network. Comput Biol Med 95: 43–54, 2018.

9. Akkus Z, Galimzianova A, Hoogi A, Rubin DL, Erickson BJ. Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions. J Digit Imaging 30: 449-59, 2017.

10. Lee JH, Kim DH, Jeong SN, Choi SH. Detection and diagnosis of dental caries using a deep learning-based convolutional neural network algorithm. J Dent 77: 106-11, 2018.

11. Krois J, Ekert T, Meinhold L, Golla T, Kharbot B, Wittemeier A, Dörfer C, Schwendicke F. Deep Learning for the Radiographic Detection of Periodontal Bone Loss. Sci Rep doi: 10.1038/s41598-019-44839-3, 2019.

12. Lee JH, Kim DH, Jeong SN, Choi SH. Diagnosis and prediction of periodontally compromised teeth using a deep learning-based convolutional neural network algorithm. J Periodontal Implant Sci 48: 114–23, 2018.

13. Poedjiastoeti W, Suebnukarn S. Application of Convolutional Neural Network in the Diagnosis of Jaw Tumors. Healthc Inform Res 24: 236-41, 2018.

14. Ariji Y, Yanashita Y, Kutsuna S, Muramatsu C, Fukuda M, Kise Y, Nozawa M, Kuwada C, Fujita H, Katsumata A, Ariji E. Automatic detection and classification of radiolucent lesions in the mandible on panoramic radiographs using a deep learning object detection technique. Oral Surg Oral Med Oral Pathol Oral Radiol 128: 424-30, 2019.

15. Kats L, Vered M, Zlotogorski-Hurvitz A, Harpaz I. Atherosclerotic carotid plaque on panoramic radiographs: neural network detection. Int J Comput Dent 22: 163-9, 2019.

16. Kats L, Vered M, Blumer S, Kats E. Neural Network Detection and Segmentation of Mental Foramen in Panoramic Imaging. J Clin Pediatr Dent 44: 168-73, 2020.

17. Kumar A, Kim J, Lyndon D, Fulham M, Feng D. An ensemble of fine-tuned convolutional neural networks for medical image classification. IEEE J. Biomed. Health Inform 21: 31–40, 2017.

18. He T, Zhang Z, Zhang H, Zhang Z, Xie J, Li M. “Bag of tricks for image classification with convolutional neural networks”. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA. doi: 10.1109/CVPR.2019.00065, 2019.

19. Kalpathy-Cramer J, A. G. S. de Herrera, Demner-Fushman D, Antani S, Bedrick S, Müller H. Evaluating performance of biomedical image retrieval systems. An overview of the medical image retrieval task at ImageCLEF 2004-2013. Computerized Medical Imaging and Graphics 39: 55-61, 2015.

20. Hassan M, Ali S, Alquhayz H, Safdar K. Developing intelligent medical image modality classification system using deep transfer learning and LDA. Sci Rep 10: 12868, 2020.

21. Valueva M.V, Nagornov N.N, Lyakhov P.A, Valuev G.V, Chervyakov

N. I. Application of the residue number system to reduce hardware costs of the convolutional neural network implementation. Mathematics and Computers in Simulation 177: 232-43, 2020.

22. Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition. https://arxiv.org/abs/1409.1556, 2015.

23. Zoph B, Vasudevan V, Shlens J, Le QV. Learning Transferable Architectures for Scalable Image Recognition. https://arxiv.org/abs/1707.07012, 2018.

24. Chollet F. “Xception: Deep Learning with Depthwise Separable Convolutions”. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA. doi: 10.1109/CVPR.2017.195, 2017.

25. Choy G, Khalilzadeh O, Michalski M, Do S, Samir AE, Pianykh OS, Geis JR, Pandharipande PV, Brink JA, Dreyer KJ. Current Applications and Future Impact of Machine Learning in Radiology. DreyerRadiology 288: 318–28, 2018.

26. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L. ImageNet Large Scale Visual Recognition Challenge IJCV 115: 211–52, 2015.

27. Shin HC, Roth HR, Gao M, Lu L, Xu Z, Nogues I, Yao J, Mollura D, Summers RM. Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning. IEEE Trans. Med. Imaging 35: 1285–98, 2016.

28. Hung K, Montalvao C, Tanaka R, Kawai T, Bornstein MM. The use and performance of artificial intelligence applications in dental and maxillofacial radiology: A systematic review. Dentomaxillofac Radiol 49: 20190107, 2020.

29. Hung K, Yeung AWK, Tanaka R, Bornstein MM. Current Applications, Opportunities, and Limitations of AI for 3D Imaging in Dental Research and Practice. Int J Environ Res Public Health 17: 4424, 2020.

30. Alom MZ, Taha TM Yakopcic C, Westberg S, Sidike P, Nasrin MS, Hasan M, Van Essen BC, Awwal AAS, Asari VK. A State-of-the-Art Survey on Deep Learning Theory and Architectures. Electronics 8: 292, 2019.

31. Zhang K, Wu J, Chen H, Lyu P. An effective teeth recognition method using label tree with cascade network structure. Comput Med Imaging Graph 68: 61-70, 2018.

32. Kise Y, Ikeda H, Fujii T, Fukuda M, Ariji Y, Fujita H, Katsumata A, Ariji E. Preliminary study on the application of deep learning system to diagnosis of Sjögren’s syndrome on CT images. Dentomaxillofac Radiol 48: 20190019, 2019.

33. Yu Y, Lin H, Meng J, Wei X, Guo H, Zhao Z. Deep Transfer Learning for Modality Classification of Medical Images. Information 8: 91, 2017.

34. Kim I, Rajaraman S, Antani S. Visual Interpretation of Convolutional Neural Network Predictions in Classifying Medical Image Modal-ities. Diagnostics (Basel) 9: 38, 2019.

35. Leite AF, Vasconcelos KF, Willems H, Jacobs R. Radiomics and Machine Learning in Oral Healthcare. Proteomics Clin Appl 14:e1900040, 2020.