Advances in Intelligent Segmentation and 3D/4D Reconstruction of Carotid Ultrasound Imaging

doi:10.37015/AUDT.2023.230015

Abstract

Abstract:

Cardiovascular disease (CVD) is one of the ten leading causes of death worldwide. Atherosclerotic disease, which can lead to myocardial infarction and stroke, is the main cause of CVD. The two main ultrasound image phenotypes used to monitor atherosclerotic load are carotid intima-media thickness (IMT) and plaque area (PA). Early segmentation and measurement methods were based on manual or threshold segmentation, snake models, etc. Usually, these methods are semi-automatic and have poor repeatability and accuracy. Segmentation of the carotid intima-media complex (IMC) and plaque in ultrasound based on artificial intelligence can achieve good accuracy. Compared with two-dimensional ultrasound, three-dimensional/four-dimensional ultrasound can provide spatial dynamic vascular information, which is helpful for doctors to evaluate. This study reviews the progress of artificial intelligence (AI) segmentation methods based on machine learning (ML) and deep learning (DL) used in the segmentation of the IMC and plaque as well as the 3D / 4D reconstruction of carotid ultrasound.

Key words: Intima-media thickness; Carotid artery plaque; Image segmentation; 3D/4D ultrasound reconstruction; Artificial intelligence

Cancan Cui, MD, Zhaojun Li, PhD, Yanping Lin, PhD. Advances in Intelligent Segmentation and 3D/4D Reconstruction of Carotid Ultrasound Imaging. Advanced Ultrasound in Diagnosis and Therapy, 2023, 7(2): 140-151.

Figures/Tables 9

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Table 1

Figure 6

Table 2

Figure 7

References 65

[1]	Vila MDM, Remeseiro B, Grau M, Elosua R, Igual L. Last advances on automatic carotid artery analysis in ultrasound images: towards deep learning. Handbook of Artificial Intelligence in Healthcare 2022;215-247.
[2]	Smitha B, Yadav D, Joseph PK. Evaluation of carotid intima media thickness measurement from ultrasound images. Med Biol Eng Comput 2022; 60:407-419. doi: 10.1007/s11517-021-02496-7 pmid: 34988763
[3]	Nithya A, Kayalvizhi R. Measurement of lower and upper IMT from Ultrasound Video Frames. Biomedical & Pharmacology Journal 2015; 8:355-364.
[4]	Gupta R, Pachauri R, Singh AK, editors. Despeckle and segmentation of carotid artery for measurement of intima-media thickness. 2019 International Conference on Signal Processing and Communication (ICSC) 2019;345-348.
[5]	Loizou CP, Pattichis CS, Pantziaris M, Tyllis T, Nicolaides A. Snakes based segmentation of the common carotid artery intima media. Medical & Biological Engineering & Computing 2007; 45:35-49.
[6]	Zhou Y, Cheng X, Xu X, Song E. Dynamic programming in parallel boundary detection with application to ultrasound intima-media segmentation. Medical Image Analysis 2013; 17:892-906. doi: 10.1016/j.media.2013.05.009 pmid: 23837965
[7]	Archana KV, Vanithamani R, editors. A review on preprocessing and segmentation techniques in carotid artery ultrasound images. Evolutionary Computing and Mobile Sustainable Networks 2022:883-897
[8]	Qian C, Su E, Yang X. Segmentation of the common carotid intima-media complex in ultrasound images using 2-D continuous max-flow and stacked sparse auto-encoder. Ultrasound Med Biol 2020; 46:3104-3124. doi: S0301-5629(20)30336-7 pmid: 32888749
[9]	Hassan M, Chaudhry A, Khan A, Iftikhar MA. Robust information gain based fuzzy c-means clustering and classification of carotid artery ultrasound images. Comput Methods Programs Biomed 2014; 113:593-609. doi: 10.1016/j.cmpb.2013.10.012
[10]	Mi S, Wei Z, Xu J, Yu Z, Yang W, Liao Q. Detecting carotid intima-media from small-sample ultrasound images. Annu Int Conf IEEE Eng Med Biol Soc 2020; 2020:2129-2132. doi: 10.1109/EMBC44109.2020.9176282 pmid: 33018427
[11]	K BJ, C R, Madian N, T S. Convolutional neural network for segmentation and measurement of intima media thickness. J Med Syst 2018; 42:1-8. doi: 10.1007/s10916-017-0844-y
[12]	Subramaniam S, Jayanthi KB, Rajasekaran C, Kuchelar R, editors. Deep learning architectures for medical image segmentation. IEEE International Symposium on Computer-Based Medical Systems 2020;579-584.
[13]	del Mar Vila M, Remeseiro B, Grau M, Elosua R, Betriu A, Fernandez-Giraldez E, et al. Semantic segmentation with DenseNets for carotid artery ultrasound plaque segmentation and CIMT estimation. Artificial Intelligence in Medicine 2020; 103:101784. doi: 10.1016/j.artmed.2019.101784
[14]	Li L, Hu Z, Huang Y, Zhu W, Wang Y, Chen M, et al. Automatic multi-plaque tracking and segmentation in ultrasonic videos. Med Image Anal 2021; 74:102201. doi: 10.1016/j.media.2021.102201
[15]	Biswas M, Saba L, Chakrabartty S, Khanna NN, Song H, Suri HS, et al. Two-stage artificial intelligence model for jointly measurement of atherosclerotic wall thickness and plaque burden in carotid ultrasound: a screening tool for cardiovascular/stroke risk assessment. Comput Biol Med 2020; 123:103847. doi: 10.1016/j.compbiomed.2020.103847
[16]	Huang Q, Zhao L, Ren G, Wang X, Liu C, Wang W. NAG-Net: nested attention-guided learning for segmentation of carotid lumen-intima interface and media-adventitia interface. Comput Biol Med 2023; 156:106718. doi: 10.1016/j.compbiomed.2023.106718
[17]	Huang QH, Jia LZ, Ren GQ, Wang XY, Liu CY. Extraction of vascular wall in carotid ultrasound via a novel boundary-delineation network. Eng Appl Artif Intel 2023; 121:106069. doi: 10.1016/j.engappai.2023.106069
[18]	Yuan YC, Li CC, Xu L, Zhu SM, Hua Y, Zhang JC. CSM-Net: automatic joint segmentation of intima-media complex and lumen in carotid artery ultrasound images. Computers in Biology and Medicine 2022; 150:106119. doi: 10.1016/j.compbiomed.2022.106119
[19]	Lin YP, Huang JH, Chen YH, Chen QQ, Li ZJ, Cao QX. Intelligent segmentation of intima-media and plaque recognition in carotid artery ultrasound images. Ultrasound in Medicine and Biology 2022; 48:469-479. doi: 10.1016/j.ultrasmedbio.2021.11.001
[20]	Azzopardi C, Camilleri KP, Hicks YA. Bimodal automated carotid ultrasound segmentation using geometrically constrained deep neural networks. Ieee J Biomed Health 2020; 24:1004-1015. doi: 10.1109/JBHI.2020.2965088 pmid: 31944969
[21]	Zhou R, Guo F, Azarpazhooh MR, Spence JD, Ukwatta E, Ding M, et al. A voxel-based fully convolution network and continuous max-flow for carotid vessel-wall-volume segmentation from 3D ultrasound images. IEEE Transactions on Medical Imaging 2020; 39:2844-2855. doi: 10.1109/TMI.2020.2975231 pmid: 32142426
[22]	Lin YP, Huang JH, Xu WJ, Cui CC, Xu WZ, Li ZJ. Method for carotid artery 3-D ultrasound image segmentation based on cswin transformer. Ultrasound in Medicine and Biology 2023; 49:645-656. doi: 10.1016/j.ultrasmedbio.2022.11.005
[23]	Zhou R, Guo FM, Azarpazhooh MR, Spence JD, Gan HT, Ding MY, et al. Carotid vessel-wall-volume ultrasound measurement via a UNet plus plus ensemble algorithm trained on small data sets. Ultrasound in Medicine and Biology 2023; 49:1031-1036. doi: 10.1016/j.ultrasmedbio.2022.12.005
[24]	Tan HC, Shi HM, Lin MQ, Spence JD, Chan KL, Chiu B. Vessel wall segmentation of common carotid artery via multi-branch light network. Proc Spie 2021; 11313:228-233.
[25]	Jiang M, Spence JD, Chiu B, editors. Segmentation of 3D ultrasound carotid vessel wall using U-Net and segmentation average network. 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) 2020;2043-2046.
[26]	Zhou R, Fenster A, Xia Y, Spence JD, Ding M. Deep learning-based carotid media-adventitia and lumen-intima boundary segmentation from three-dimensional ultrasound images. Medical physics 2019; 46:3180-3193. doi: 10.1002/mp.13581 pmid: 31071228
[27]	Qian CJ, Yang XP. An integrated method for atherosclerotic carotid plaque segmentation in ultrasound image. Comput Meth Prog Bio 2018; 153:19-32. doi: S0169-2607(17)30307-3 pmid: 29157451
[28]	Qian CJ, Su EJ, Ni XY. Learning-based initialization for correntropy-based level sets to segment atherosclerotic plaque in ultrasound images. Ultrasonics 2023; 127:106826. doi: 10.1016/j.ultras.2022.106826
[29]	Nagaraj Y, Teja AHS, Narasimhadhan AV. Automatic segmentation of intima media complex in carotid ultrasound images using support vector machine. Arab J Sci Eng 2019; 44:3489-3496. doi: 10.1007/s13369-018-3549-8
[30]	Zhou R, Guo FM, Azarpazhooh MR, Hashemi S, Cheng XY, Spence JD, et al. Deep learning-based measurement of total plaque area in B-mode ultrasound images. Ieee J Biomed Health 2021; 25:2967-2977.
[31]	Zhou R, Wang FR, Fang XY, Fenster A, Gan HT. An adaptively weighted ensemble of multiple CNNs for carotid ultrasound image segmentation. Biomed Signal Proces 2023; 83:104673. doi: 10.1016/j.bspc.2023.104673
[32]	Kostelansky M, Rodriguez AM, Kybic J, Hekrdla M, Dvorsky O, Kozel J, et al. Differentiating between stable and progressive carotid atherosclerotic plaques from in-vivo ultrasound images using texture descriptors. 17th International Symposium on Medical Information Processing and Analysis 2021; 12088:511-520.
[33]	Mi SY, Bao QQ, Wei ZH, Xu F, Yang WM. MBFF-Net: Multi-branch feature fusion network for carotid plaque segmentation in ultrasound. Lect Notes Comput Sc 2021; 12905:313-322.
[34]	Li YH, Zou L, Xiong L, Yu F, Jiang H, Fan C, et al. FRDD-Net: automated carotid plaque ultrasound images segmentation using feature remapping and dense decoding. Sensors-Basel 2022; 22:887. doi: 10.3390/s22030887
[35]	Jain PK, Sharma N, Giannopoulos AA, Saba L, Nicolaides A, Suri JS. Hybrid deep learning segmentation models for atherosclerotic plaque in internal carotid artery B-mode ultrasound. Computers in Biology and Medicine 2021; 136:104721. doi: 10.1016/j.compbiomed.2021.104721
[36]	Yuan YC, Li CC, Zhang K, Hua Y, Zhang JC. HRU-Net: a transfer learning method for carotid artery plaque segmentation in ultrasound images. Diagnostics 2022; 12:2852. doi: 10.3390/diagnostics12112852
[37]	Zhou R, Ou YH, Fang XY, Azarpazhooh MR, Gan HT, Ye ZW, et al. Ultrasound carotid plaque segmentation via image reconstruction-based self-supervised learning with limited training labels. Math Biosci Eng 2023; 20:1617-1636. doi: 10.3934/mbe.2023074
[38]	Xie M, Li Y, Xue Y, Huntress L, Roshan UW, editors. Two-stage and dual-decoder convolutional U-Net ensembles for reliable vessel and plaque segmentation in carotid ultrasound images. 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA) 2020;1376-1381.
[39]	Zhou R, Ma W, Fenster A, Ding M, editors. U-Net based automatic carotid plaque segmentation from 3D ultrasound images. Medical Imaging 2019: Computer-Aided Diagnosis 2019; 10950:1119-1125.
[40]	Moon, Hyungil, Geonhwan, Shin, Hayong, Park, et al. 3D freehand ultrasound reconstruction using a piecewise smooth Markov random field. Computer vision and image understanding: CVIU 2016; 151:101-113. doi: 10.1016/j.cviu.2015.12.009
[41]	Zhenping, Chen, Qinghua, Huang. Real-time freehand 3D ultrasound imaging. Computer Methods in Biomechanics & Biomedical Engineering Imaging & Visualization 2018; 6:74-83.
[42]	De Ruijter J, Muijsers JJ, Van de Vosse FN, Van Sambeek MR, Lopata RG. A generalized approach for automatic 3-D geometry assessment of blood vessels in transverse ultrasound images using convolutional neural networks. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control 2021; 68:3326-3335. doi: 10.1109/TUFFC.2021.3090461
[43]	Li J, Huang Y, Song S, Chen H, Shi J, Xu D, et al. Automatic diagnosis of carotid atherosclerosis using a portable freehand 3D ultrasound imaging system. arXiv preprint arXiv 230103081 2023.
[44]	Meza J, Contreras-Ortiz SH, Romero LA, Marrugo AG. Three-dimensional multimodal medical imaging system based on freehand ultrasound and structured light. Opt Eng 2021; 60:054106.
[45]	Huang Q, Lan J, Li X. Robotic arm based automatic ultrasound scanning for three-dimensional imaging. IEEE Transactions on Industrial Informatics 2019; 15:1173-1182. doi: 10.1109/TII.2018.2871864
[46]	Qi J, Ding M, Yuchi M, editors. 3D ultrasound data acqusition system based on back end scan mode. 2011:156:158.
[47]	Housden RJ, Gee AH, Treece GM, Prager RW. Sensorless reconstruction of unconstrained freehand 3D ultrasound data. Ultrasound in Medicine and Biology 2007; 33:408-419. pmid: 17280771
[48]	Guo H, Xu S, Wood B, Yan P, editors. Sensorless freehand 3D ultrasound reconstruction via deep contextual learning. Medical Image Computing and Computer Assisted Intervention-MICCAI 2020: 23rd International Conference 2020;463-472
[49]	Groves LA, VanBerlo B, Veinberg N, Alboog A, Peters TM, Chen ECS. Automatic segmentation of the carotid artery and internal jugular vein from 2D ultrasound images for 3D vascular reconstruction. Int J Comput Ass Rad 2020; 15:1835-1846.
[50]	Prevost R, Salehi M, Sprung J, Ladikos A, Bauer R, Wein W, editors. Deep learning for sensorless 3D freehand ultrasound imaging. Medical Image Computing and Computer-Assisted Intervention- MICCAI 2017: 20th International Conference 2017;628-636
[51]	Prevost R, Salehi M, Jagoda S, Kumar N, Sprung J, Ladikos A, Bauer R, Zettinig O, Wein W. 3D freehand ultrasound without external tracking using deep learning. Med Image Anal 2018; 48:187-202. doi: S1361-8415(18)30371-2 pmid: 29936399
[52]	Djukic T, Arsic B, Djorovic S, Filipovic N, Koncar I. Validation of the machine learning approach for 3D reconstruction of carotid artery from ultrasound imaging. Ieee Int C Bioinf Bi 2020;789-794.
[53]	Luo MY, Yang X, Huang XQ, Huang YH, Zou YX, Hu XD, et al. Self context and shape prior for sensorless freehand 3D ultrasound reconstruction. Medical Image Computing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference 2021; 12906:201-210.
[54]	Luo MY, Yang X, Wang HZ, Du LW, Ni D. Deep motion network for freehand 3D ultrasound reconstruction. Medical Image Computing and Computer Assisted Intervention-MICCAI 2022: 25th International Conference 2022; 13434:290-299.
[55]	Meairs S, Hennerici M. Four-dimensional ultrasonographic characterization of plaque surface motion in patients with symptomatic and asymptomatic carotid artery stenosis. Stroke 1999; 30:1807-1813. pmid: 10471428
[56]	Pace DF, Gobbi DG, Wedlake C, Gumprecht J, Peters M. An open-source real-time ultrasound reconstruction system for four-dimensional imaging of moving organs. The MIDAS Journal 2009.
[57]	Birgelen CV, Mintz GS, Nicosia A, Foley DP, Van D, Bruining N, et al. Electrocardiogram-gated intravascular ultrasound image acquisition after coronary stent deployment facilitates on-line three-dimensional reconstruction and automated lumen quantification. Journal of the American College of Cardiology 1997; 30:436-443. pmid: 9247516
[58]	Thurgood H, Witte R, Laksari K. 4D Reconstruction and identification of carotid artery stenosis utilizing a novel pulsatile ultrasound phantom. Current Protocols 2021; 1:e264.
[59]	Chittajallu DR, McCormick M, Gerber S, Czernuszewicz TJ, Gessner R, Willis MS, et al. Image-based methods for phase estimation, gating, and temporal superresolution of cardiac ultrasound. Ieee T Bio-Med Eng 2019; 66:72-79. doi: 10.1109/TBME.2018.2823279
[60]	Fu TY, Li JS, Zhang JJ, Ai DN, Fan JF, Song H, et al. Four-dimensional wide-field ultrasound reconstruction system with sparse respiratory signal matching. Ieee T Comput Imag 2021; 7:234-247.
[61]	Panayiotou M, King AP, Housden RJ, Ma YL, Cooklin M, O'Neill M, et al. A statistical method for retrospective cardiac and respiratory motion gating of interventional cardiac x-ray images. Medical Physics 2014; 41:071901. doi: 10.1118/1.4881140
[62]	Wachinger C, Yigitsoy M, Rijkhorst EJ, Navab N. Manifold learning for image-based breathing gating in ultrasound and MRI. Medical Image Analysis 2012; 16:806-818. doi: 10.1016/j.media.2011.11.008 pmid: 22226466
[63]	Wachinger C, Yigitsoy M, Navab N. Manifold learning for image-based breathing gating with application to 4D ultrasound. Medical Image Computing and Computer-Assisted Intervention-MICCAI 2010: 13th International Conference 2010; 6362:26-33.
[64]	Liang HY, Ning GC, Dai SQ, Ma LF, Luo JW, Zhang XR, et al. Spatiotemporal reconstruction method of carotid artery ultrasound from freehand sonography. Int J Comput Ass Rad 2022; 17:1731-1743.
[65]	Dezaki FT, Liao ZB, Luong C, Girgis H, Dhungel N, Abdi AH, et al. Cardiac phase detection in echocardiograms with densely gated recurrent neural networks and global extrema loss. Ieee Transactions on Medical Imaging 2019; 38:1821-1832. doi: 10.1109/TMI.2018.2888807 pmid: 30582532

Author	Year	Type	Images number	Method	Performance
Huang et al. [16]	2023	longitudinal section	186	NAG-Net	DSC = 89.9% JSC = 81.7% ACC = 90.3%
Yuan et al. [18]	2022	longitudinal section	2019	CSM-Net	DSC = 81.4% Precision = 91.1% Recall = 91.6%
Jayanthi et al. [11]	2018	longitudinal section	220	CNN + Snake	Mean error = 0.08 mm
Azzopardi et al. [20]	2020	cross section	81000	UNet + geometric constraint	DSC = 96.2% MHD = 0.192 mm
Lin et al. [22]	2023	3D	213	U-CSWT	DSC = 94.6% MSD = 0.10 mm
Zhou et al. [21]	2020	3D	1007	Voxel-FCN	DSC = 93.2% MAD = 0.30 mm

Author	Year	Type	Images number	Method	Result
Zhou et al. [31]	2023	longitudinal section	510	Adaptive combination of multiple CNNS	DSC = 88.9% HD = 0.91 mm
Jain et al. [35]	2022	longitudinal section	970	SegNet-UNet+	DSC = 89.5% JI = 81.2%
Mi et al. [33]	2021	longitudinal section	430	MBFF-Net	DSC = 78.0% IoU = 70.2%
Zhou et al. [30]	2021	longitudinal section	497	UNet++ ensemble	DSC = 88.6%
Qian et al. [27]	2018	longitudinal section	29	Random forest+auto-context	DSC = 81.0% HD = 1.75 mm
Xie et al. [38]	2020	cross section	500	Dual-decoder U-Net	DSC = 87.3%
Zhou et al. [39]	2019	3D	34	Unsupervised learning+U-Net	DSC = 90.7% MAD = 0.15 mm