Article Data

  • Views 530
  • Dowloads 163

Original Research

Open Access

Co-Mask R-CNN: collaborative learning-based method for tooth instance segmentation

  • Chen Wang1
  • Jingyu Yang1
  • Hongzhi Liu1
  • Peng Yu2,*,
  • Xijun Jiang3,*,
  • Ruijun Liu4

1The School of Computer and Artificial Intelligence, Beijing Technology and Business University, 100048 Beijing, China

2Department of Cariology and Endodontology, Peking University School and Hospital of Stomatology, 100081 Beijing, China

3Department of Pediatric Dentistry, Peking University School and Hospital of Stomatology, 100081 Beijing, China

4The School of Software, Beihang University, 100191 Beijing, China

DOI: 10.22514/jocpd.2024.136 Vol.48,Issue 6,November 2024 pp.161-172

Submitted: 21 December 2023 Accepted: 04 March 2024

Published: 03 November 2024

*Corresponding Author(s): Peng Yu E-mail: yupeng@bjmu.edu.cn
*Corresponding Author(s): Xijun Jiang E-mail: jiangxijun@bjmu.edu.cn

Abstract

Traditional tooth image analysis methods primarily focus on feature extraction from individual images, often overlooking critical tooth shape and position information. This paper presents a novel computer-aided diagnosis method, Collaborative learning with Mask Region-based Convolutional Neural Network (Co-Mask R-CNN), designed to enhance tooth image analysis by leveraging the integration of complementary information. First, image enhancement is employed to generate an edge-enhanced tooth edge image. Then, a collaborative learning strategy combined with Mask R-CNN is introduced, where the original and edge images are input simultaneously, and a two-stream encoder extracts feature maps from complementary images. By utilizing an attention mechanism, the output features from the two branches are dynamically fused, quantifying the relative importance of the two complementary images at different spatial positions. Finally, the fused feature map is utilized for tooth instance segmentation. Extensive experiments are conducted using a proprietary dataset to evaluate the effectiveness of Co-Mask R-CNN, and the results are compared against those of an alternative segmentation network. The results demonstrate that Co-Mask R-CNN outperforms the other networks in terms of both segmentation accuracy and robustness. Consequently, this method holds considerable promise for providing medical professionals with precise tooth segmentation results, establishing a reliable foundation for subsequent tooth disease diagnosis and treatment.


Keywords

Deep learning; Two-stream collaborative network; Tooth instance segmentation; Bitewing radiograph


Cite and Share

Chen Wang,Jingyu Yang,Hongzhi Liu,Peng Yu,Xijun Jiang,Ruijun Liu. Co-Mask R-CNN: collaborative learning-based method for tooth instance segmentation. Journal of Clinical Pediatric Dentistry. 2024. 48(6);161-172.

References

[1] Kumar R, Saha P. A review on artificial intelligence and machine learning to improve cancer management and drug discovery. International Journal for Research in Applied Sciences and Biotechnology. 2022; 9: 149–156.

[2] Huang C, Wang J, Wang S, Zhang Y. A review of deep learning in dentistry. Neurocomputing. 2023; 554: 126629.

[3] Helli S, Hamamci A. Tooth instance segmentation on panoramic dental radiographs using u-nets and morphological processing. Duzce University Journal of Science and Technology. 2022; 10: 39–50.

[4] Martins M V, Baptista L, Luís H, Assunção V, Araújo MR, Realinho V. Machine learning in x-ray diagnosis for oral health: a review of recent progress. Computation. 2023; 11: 115.

[5] Zhang J, Jiang Y, Gao F, Zhao S, Yang F, Song L. A fast automatic reconstruction method for panoramic images based on cone beam computed tomography. Electronics. 2022; 11: 2404.

[6] Polizzi A, Quinzi V, Ronsivalle V, Venezia P, Santonocito S, Giudice A Lo, et al. Tooth automatic segmentation from CBCT images: a systematic review. Clinical Oral Investigations. 2023: 27: 3363–3378.

[7] Bruellmann D, Sander S, Schmidtmann I. The design of an fast fourier filter for enhancing diagnostically relevant structures—endodontic files. Computers in Biology and Medicine. 2016; 72: 212–217.

[8] Oltu B, Karaca BK, Uyar T, Uyar DS. Detection of occlusal plaque and caries using fuzzy C means based segmentation algorithm. 2021 International Conference on INnovations in Intelligent SysTems and Applications (INISTA) (pp. 1–5). IEEE. 2021.

[9] Wang C, Huang C, Lee J, Li C, Chang S, Siao M, et al. A benchmark for comparison of dental radiography analysis algorithms. Medical Image Analysis. 2016; 31: 63–76.

[10] Patil S, Kulkarni V, Bhise A. Algorithmic analysis for dental caries detection using an adaptive neural network architecture. Heliyon. 2019; 5: e01579.

[11] Yau H, Yang T, Chen Y. Tooth model reconstruction based upon data fusion for orthodontic treatment simulation. Computers in Biology and Medicine. 2014; 48: 8–16.

[12] Wang Y, Liu S, Wang G, Liu Y. Accurate tooth segmentation with improved hybrid active contour model. Physics in Medicine and Biology. 2018; 64: 015012.

[13] Salimzadeh S, Kandulu S. Teeth segmentation of bitewing X-ray images using wavelet transform. Informatica. 2020; 44: 421–426.

[14] Modi CK, Desai NP. A simple and novel algorithm for automatic selection of ROI for dental radiograph segmentation. 2011 24th Canadian Conference on Electrical and Computer Engineering (CCECE) (pp. 000504–000507). IEEE. 2011.

[15] He K, Gkioxari G, Dollar P, Girshick R. Mask R-CNN. 2017 IEEE International Conference on Computer Vision. 2017; 2961–2969.

[16] Jader G, Fontineli J, Ruiz M, Abdalla K, Pithon M, Oliveira L. Deep instance segmentation of teeth in panoramic X-ray images. 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). 2018; 400–407.

[17] Pinheiro L, Silva B, Sobrinho B, Lima F, Cury P, Oliveira L. Numbering permanent and deciduous teeth via deep instance segmentation in panoramic X-rays. 17th International Symposium on Medical Information Processing and Analysis. 2021; 12088: 95–104.

[18] AlQarni S, Chandrashekar G, Bumann EE, Lee Y. Incremental learning for panoramic radiograph segmentation. 2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). 2022; 557–561.

[19] Ren S, He K, Girshick R, Sun J. Faster R-CNN: towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems. 2015; 28.

[20] Lee J, Han S, Kim YH, Lee C, Kim I. Application of a fully deep convolutional neural network to the automation of tooth segmentation on panoramic radiographs. Oral Surgery, Oral Medicine, Oral Pathology, and Oral Radiology. 2020; 129: 635–642.

[21] Zhao Y, Li P, Gao C, Liu Y, Chen Q, Yang F, et al. TSASNet: tooth segmentation on dental panoramic X-ray images by two-stage attention segmentation network. Knowledge-Based Systems. 2020; 206: 106338.

[22] Chung M, Lee M, Hong J, Park S, Lee J, Lee J, et al. Pose-aware instance segmentation framework from cone beam CT images for tooth segmentation. Computers in Biology and Medicine. 2020; 120: 103720.

[23] Silva B, Pinheiro L, Oliveira L, Pithon M. A study on tooth segmentation and numbering using end-to-end deep neural networks. 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI). 2020; 164–171.

[24] Wang K, Liew JH, Zou Y, Zhou D, Feng J. PANet: few-shot image semantic segmentation with prototype alignment. 2019 IEEE/CVF International Conference on Computer Vision (ICCV). 2019; 9197–9206.

[25] Zhang H, Wu C, Zhang Z, Zhu Y, Lin H, Zhang Z, et al. ResNeSt: split-attention networks. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 2022; 2736–2746.

[26] Chen K, Pang J, Wang J, Xiong Y, Li X, Sun S, et al. Hybrid task cascade for instance segmentation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019; 4974–4983.

[27] Leite AF, Gerven AV, Willems H, Beznik T, Lahoud P, Gaêta-Araujo H, et al. Artificial intelligence-driven novel tool for tooth detection and segmentation on panoramic radiographs. Clinical Oral Investigations. 2021; 25: 2257–2267.

[28] Fu J, Liu J, Tian H, Li Y, Bao Y, Fang Z, et al. Dual attention network for scene segmentation. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019; 3146–3154.

[29] Zhou Y, He X, Huang L, Liu L, Zhu F, Cui S, et al. Collaborative learning of semi-supervised segmentation and classification for medical images. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2019; 2079–2088.

[30] Kumar A, Fulham M, Feng D, Kim J. Co-learning feature fusion maps from pet-CT images of lung cancer. IEEE Transactions on Medical Imaging. 2020; 39: 204–217.

[31] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016; 770–778.

[32] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C, et al. SSD: single shot multibox detector. Computer Vision—ECCV 2016. 2016; 104: 21–37.

[33] Redmon J, Farhadi A. Yolov3: An incremental improvement[J]. arXiv preprint arXiv:1804.02767, 2018.

[34] Wang C, Bochkovskiy A, Liao HM. YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2023; 7464–7475.


Abstracted / indexed in

Science Citation Index Expanded (SciSearch) Created as SCI in 1964, Science Citation Index Expanded now indexes over 9,500 of the world’s most impactful journals across 178 scientific disciplines. More than 53 million records and 1.18 billion cited references date back from 1900 to present.

Biological Abstracts Easily discover critical journal coverage of the life sciences with Biological Abstracts, produced by the Web of Science Group, with topics ranging from botany to microbiology to pharmacology. Including BIOSIS indexing and MeSH terms, specialized indexing in Biological Abstracts helps you to discover more accurate, context-sensitive results.

Google Scholar Google Scholar is a freely accessible web search engine that indexes the full text or metadata of scholarly literature across an array of publishing formats and disciplines.

JournalSeek Genamics JournalSeek is the largest completely categorized database of freely available journal information available on the internet. The database presently contains 39226 titles. Journal information includes the description (aims and scope), journal abbreviation, journal homepage link, subject category and ISSN.

Current Contents - Clinical Medicine Current Contents - Clinical Medicine provides easy access to complete tables of contents, abstracts, bibliographic information and all other significant items in recently published issues from over 1,000 leading journals in clinical medicine.

BIOSIS Previews BIOSIS Previews is an English-language, bibliographic database service, with abstracts and citation indexing. It is part of Clarivate Analytics Web of Science suite. BIOSIS Previews indexes data from 1926 to the present.

Journal Citation Reports/Science Edition Journal Citation Reports/Science Edition aims to evaluate a journal’s value from multiple perspectives including the journal impact factor, descriptive data about a journal’s open access content as well as contributing authors, and provide readers a transparent and publisher-neutral data & statistics information about the journal.

Scopus: CiteScore 1.8 (2023) Scopus is Elsevier's abstract and citation database launched in 2004. Scopus covers nearly 36,377 titles (22,794 active titles and 13,583 Inactive titles) from approximately 11,678 publishers, of which 34,346 are peer-reviewed journals in top-level subject fields: life sciences, social sciences, physical sciences and health sciences.

Submission Turnaround Time

Conferences

Top