Deep Learning in Ultrasound Localization Microscopy

doi:10.37015/AUDT.2024.240023

Abstract

Abstract:

Ultrasound imaging holds a significant position in medical diagnostics due to its non-invasive and real-time capabilities. However, traditional ultrasound is constrained by the diffraction limit, making it challenging to capture fine blood vessels. Ultrasound localization microscopy (ULM) overcomes this limitation by achieving super-resolution imaging through tracking the trajectories of microbubbles (MBs) within microvasculature. This review summarizes the applications of deep learning (DL) techniques in ULM post-processing algorithms, including key steps such as beamforming, clutter filtering and denoising, localization, and tracking. Although DL shows great potential in improving ULM imaging quality and efficiency, current research mainly focuses on imaging algorithmic improvements rather than in-depth image analysis. In the future, with the accumulation of ULM image data, the powerful feature extraction capability of DL is expected to further advance ULM applications in disease prediction and diagnosis.

Key words: Ultrasound localization microscopy; Deep learning; Ultrasound microvasculomics

Yuhang Zheng, BS, Jianqiao Zhou, MD. Deep Learning in Ultrasound Localization Microscopy. Advanced Ultrasound in Diagnosis and Therapy, 2024, 8(3): 86-92.

Figures/Tables 6

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

References 51

[1]	Phenix CP, Togtema M, Pichardo S, Zehbe I, Curiel L. High intensity focused ultrasound technology, its scope and applications in therapy and drug delivery. J Pharm Pharm Sci 2014;17:136-153.
[2]	Fu WX, Wang Q, Zhang YS, Li Y, Xu T, He S, et al. Application of ultrasound technology in the diagnosis and treatment of digestive tract diseases. Eur Rev Med Pharmacol Sci 2015;19:602-606.
[3]	Chavignon A, Heiles B, Hingot V, Orset C, Vivien D, Couture O. 3D Transcranial Ultrasound Localization Microscopy in the Rat Brain With a Multiplexed Matrix Probe. IEEE Trans Biomed Eng 2022;69:2132-2142.
[4]	Zhang C, Lei S, Ma A, Wang B, Wang S, Liu J, et al. Evaluation of tumor microvasculature with 3D ultrasound localization microscopy based on 2D matrix array. Eur Radiol 2024;34:5250-5259.
[5]	Huang C, Zhang W, Gong P, Lok UW, Tang S, Yin T, et al. Super-resolution ultrasound localization microscopy based on a high frame-rate clinical ultrasound scanner: an in-human feasibility study. Phys Med Biol 2021;66: 10.1088/1361-6560/abef45.
[6]	Errico C, Pierre J, Pezet S, Desailly Y, Lenkei Z, Couture O, et al. Ultrafast ultrasound localization microscopy for deep super-resolution vascular imaging. Nature. 2015;527:499-502.
[7]	Kim J, Dong Z, Lowerison MR, Sekaran NVC, You Q, Llano DA, et al. Deep Learning-based 3D Beamforming on a 2D Row Column Addressing (RCA) Array for 3D Super-resolution Ultrasound Localization Microscopy. 2022 IEEE International Ultrasonics Symposium (IUS); Venice, Italy, 2022, pp. 1-4
[8]	Christensen-Jeffries K, Couture O, Dayton PA, Eldar YC, Hynynen K, Kiessling F, et al. Super-resolution Ultrasound Imaging. Ultrasound Med Biol 2020;46:865-891.
[9]	van Sloun RJG, Solomon O, Bruce M, Khaing ZZ, Wijkstra H, Eldar YC, et al. Super-resolution ultrasound localization microscopy through deep learning. IEEE Trans Med Imaging 2021;40:829-839..
[10]	Ackermann D, Schmitz G. Detection and tracking of multiple microbubbles in ultrasound B-mode images. IEEE Trans Ultrason Ferroelectr Freq Control 2016;63:72-82.
[11]	Jiang Y, Yang M, Wang S, Li X, Sun Y. Emerging role of deep learning-based artificial intelligence in tumor pathology. Cancer Commun (Lond) 2020;40:154-166.
[12]	van der Velden BHM, Kuijf HJ, Gilhuijs KGA, Viergever MA. Explainable artificial intelligence (XAI) in deep learning-based medical image analysis. Med Image Anal 2022;79:102470.
[13]	Cui R, Yu H, Xu T, Xing X, Cao X, Yan K, et al. Deep Learning in Medical Hyperspectral Images: A Review. Sensors (Basel). 2022;22:9790.
[14]	Cammarasana S, Nicolardi P, Patane G. Real-time denoising of ultrasound images based on deep learning. Med Biol Eng Comput 2022;60:2229-2244.
[15]	Abbasian Ardakani A, Mohammadi A, Vogl TJ, Kuzan TY, Acharya UR. AdaRes: A deep learning-based model for ultrasound image denoising: Results of image quality metrics, radiomics, artificial intelligence, and clinical studies. J Clin Ultrasound 2024;52:131-143.
[16]	Gao Y, Zeng S, Xu X, Li H, Yao S, Song K, et al. Deep learning-enabled pelvic ultrasound images for accurate diagnosis of ovarian cancer in China: a retrospective, multicentre, diagnostic study. Lancet Digit Health 2022;4:e179-e187.
[17]	Yang X, Chen Z, Jia X. Deep Learning Algorithm-Based Ultrasound Image Information in Diagnosis and Treatment of Pernicious Placenta Previa. Comput Math Methods Med 2022;2022:3452176.
[18]	Cai R, Liu Y, Sun Z, Wang Y, Wang Y, Li F, Jiang H. Deep-learning based segmentation of ultrasound adipose image for liposuction. Int J Med Robot 2023;19:e2548.
[19]	Marzola F, van Alfen N, Doorduin J, Meiburger KM. Deep learning segmentation of transverse musculoskeletal ultrasound images for neuromuscular disease assessment. Comput Biol Med 2021;135:104623.
[20]	Xing P, Poree J, Rauby B, Malescot A, Martineau E, Perrot V, et al. Phase aberration correction for in vivo ultrasound localization microscopy using a spatiotemporal complex-valued neural network. IEEE Trans Med Imaging 2024;43:662-673.
[21]	Brown KG, Ghosh D, Hoyt K. Deep learning of spatiotemporal filtering for fast super-resolution ultrasound imaging. IEEE Trans Ultrason Ferroelectr Freq Control 2020;67:1820-1829.
[22]	Santos CAN, Martins DLN, Mascarenhas NDA. Ultrasound image despeckling using stochastic distance-based BM3D. IEEE Trans Image Process 2017;26:2632-2643.
[23]	R. Wang, W. -N. Lee. A General Deep Learning Model for Ultrasound Localization Microscopy. IEEE International Ultrasonics Symposium (IUS) Venice, Italy, 2022, pp. 1-4
[24]	Yu X, Luan S, Lei S, Huang J, Liu Z, Xue X, et al. Deep learning for fast denoising filtering in ultrasound localization microscopy. Phys Med Biol 2023;68.
[25]	Sloun RJGv, Solomon O, Bruce M, Khaing ZZ, Eldar YC, Mischi M. Deep Learning for Super-resolution Vascular Ultrasound Imaging. Paper presented at: ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2019:12-17.
[26]	Stevens TSW, Herbst EB, Luijten B, Ossenkoppele BW, Voskuil TJ, Wang S, et al. A hybrid deep learning pipeline for improved ultrasound localization microscopy. 2022 IEEE International Ultrasonics Symposium (IUS); Venice, Italy, 2022, pp. 1-4.
[27]	Chen X, Lowerison MR, Dong Z, Sekaran NVC, Zhang W, Llano DA, et al. Deep learning based microbubble localization for fast and robust ultrasound localization microscopy. Paper presented at: 2020 IEEE International Ultrasonics Symposium (IUS); 2020:7-11.
[28]	Liu X, Zhou T, Lu M, Yang Y, He Q, Luo J. Deep Learning for ultrasound localization microscopy. IEEE Trans Med Imaging 2020;39:3064-3078.
[29]	Youn J, Luijten B, Stuart MB, Eldar YC, van Sloun RJG, Jensen JA. Deep learning models for fast ultrasound localization microscopy. Paper presented at: 2020 IEEE International Ultrasonics Symposium (IUS); 2020:7-11.
[30]	Zhao Y, Liu S, Luo A, Peng B. Dual generative adversarial network for ultrasound localization microscopy. Paper presented at: 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC); 2022:9-12.
[31]	Gu W, Yan Z, Li B, Liu C, Ta D, Liu X. GAN-Based Ultrasound Localization Microscopy. Paper presented at: 2022 IEEE International Ultrasonics Symposium (IUS); 10-13 Oct. 2022, 2022.
[32]	Lok UW, Huang C, Gong P, Tang S, Yang L, Zhang W, et al. Fast super-resolution ultrasound microvessel imaging using spatiotemporal data with deep fully convolutional neural network. Phys Med Biol. 2021;66:10.1088/1361-6560/abeb31.
[33]	Luan S, Yu X, Lei S, Ma C, Wang X, Xue X, et al. Deep learning for fast super-resolution ultrasound microvessel imaging. Phys Med Biol. 2023;68.
[34]	Youn J, Luijten B, Schou M, Stuart MB, Eldar YC, van Sloun RJG, et al. Model-based deep learning on ultrasound channel data for fast ultrasound localization microscopy. Paper presented at: 2021 IEEE International Ultrasonics Symposium (IUS); 2021:11-16.
[35]	Zhou T, Lu M, Yang Y, He Q, Luo J, Liu X. qULM-DL: Quantitative ultrasound localization microscopy via deep learning. Paper presented at: 2020 IEEE International Ultrasonics Symposium (IUS); 2020:7-11.
[36]	Hahne C, Chabouh G, Chavignon A, Couture O, Sznitman R. RF-ULM: Ultrasound localization microscopy learned from radio-frequency wavefronts. IEEE Trans Med Imaging. 2024;43:3253-3262.
[37]	Shin Y, Lowerison MR, Wang Y, Chen X, You Q, Dong Z, et al. Context-aware deep learning enables high-efficacy localization of high concentration microbubbles for super-resolution ultrasound localization microscopy. Nat Commun 2024;15:2932.
[38]	Blanken N, Wolterink JM, Delingette H, Brune C, Versluis M, Lajoinie G. Super-resolved microbubble localization in single-channel ultrasound RF signals using deep learning. IEEE Trans Med Imaging 2022;41:2532-2542.
[39]	Liu X, Almekkawy M. Ultrasound localization microscopy using deep neural network. IEEE Trans Ultrason Ferroelectr Freq Control 2023;70:625-635.
[40]	Liu X, Almekkawy M. Ultrasound super resolution using deep learning based on attention mechanism. Paper presented at: 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI); 2023:18-21.
[41]	Liu X, Almekkawy M. Ultrasound Super resolution using vision transformer with convolution projection operation. Paper presented at: 2022 IEEE International Ultrasonics Symposium (IUS); 2022:10-13.
[42]	Zhang Z, Hwang M, Kilbaugh TJ, Katz J. Improving sub-pixel accuracy in ultrasound localization microscopy using supervised and self-supervised deep learning. Meas Sci Technol 2024;35:045701.
[43]	Revach G, Shlezinger N, Ni X, Escoriza AL, van Sloun RJG, Eldar YC. KalmanNet: Neural network aided kalman filtering for partially known dynamics. IEEE Transactions on Signal Processing 2022;70:1532-1547.
[44]	Milecki L, Poree J, Belgharbi H, Bourquin C, Damseh R, Delafontaine-Martel P, et al. A deep learning framework for spatiotemporal ultrasound localization microscopy. IEEE Trans Med Imaging 2021;40:1428-1437.
[45]	Zhang G, Yue Y, Dai F, Liu X, Ta D. Transformer for ultrafast ultrasound localization microscopy. Paper presented at: 2022 IEEE International Ultrasonics Symposium (IUS); 2022:10-13.
[46]	Zhang G, Gu W, Yue Y, Tang MX, Luo J, Liu X, et al. ULM-MbCNRT: In vivo ultrafast ultrasound localization microscopy by combining multi-branch CNN and recursive transformer. IEEE Trans Ultrason Ferroelectr Freq Control 2024;PP.
[47]	Zhang J, Zhang X, Yang Y, He Q, Luo J. A deep learning method for reduction of microbubble accumulation time in ultrasound localization microscopy. Paper presented at: 2020 IEEE International Ultrasonics Symposium (IUS); 2020:7-11.
[48]	Zhang G, Li B, Liu X, Ta D. A novel method for ultrasound localization microscopy based on temporal correlation through deep learning. Paper presented at: 2023 IEEE International Ultrasonics Symposium (IUS); 2023:3-8.
[49]	Opacic T, Dencks S, Theek B, Piepenbrock M, Ackermann D, Rix A, Lammers T, et al. Motion model ultrasound localization microscopy for preclinical and clinical multiparametric tumor characterization. Nature Communications 2018;9:1527.
[50]	Zhu J, Zhang C, Christensen-Jeffries K, Zhang G, Harput S, Dunsby C, et al. Super-resolution ultrasound localization microscopy of microvascular structure and flow for distinguishing metastatic lymph nodes - an initial human study. Ultraschall Med 2022;43:592-598.
[51]	Xia S, Zheng Y, Hua Q, et al. Super-resolution ultrasound and microvasculomics: a consensus statement. Eur Radiol 2024.