Implementation of a VGG-19 and Discrete Wavelet Transform Combined Multimodal Fusion Technique
Received: 30 April 2025 | Revised: 11 May 2025 and 26 May 2025 | Accepted: 31 May 2025 | Online: 15 July 2025
Corresponding author: Hayath T M
Abstract
The integration of diverse medical imaging modalities facilitates the identification of diseases. Medical imaging is a critical component of medical research and diagnosis, providing detailed information about the structure and function of the body. In some cases, imaging approaches that utilize a single modality may not capture the complete set of the diagnostic data necessary for reliable physician evaluations. The objective of this study is to enhance the clarity of medical imagery and facilitate more precise disease identification. The proposed approach involves a multimodal medical image fusion technique that integrates Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) data. The suggested technique involves three sequential steps: image registration, image merging, and image segmentation. Image registration is a process that aligns CT and MRI images by utilizing procedures that are based on landmarks to ensure that pixel-level correlation is maintained. To preserve both structural and functional characteristics from the input pictures, the fusion procedure makes use of deep learning-based transfer learning in conjunction with the VGG-19 network and Discrete Wavelet Transform (DWT). Lastly, the watershed algorithm is employed to extract and highlight Regions of Interest (ROIs), such as tumors, during the segmentation process. The suggested method substantially increases picture clarity, maintains essential characteristics, and boosts the precision of tumor segmentation, as demonstrated by the results of the experiments.
Keywords:
medical image fusion, Computed Tomography (CT), Magnetic Resonance Imaging (MRI), disease detection, image registartion, VGG-19, Discrete Wavelet Transform (DWT), segmentation algorithmsReferences
M. A. Saleh, A. A. Ali, K. Ahmed, and A. M. Sarhan, "A Brief Analysis of Multimodal Medical Image Fusion Techniques," Electronics, vol. 12, no. 1, Jan. 2023, Art. no. 97. DOI: https://doi.org/10.3390/electronics12010097
H. Kaur, D. Koundal, and V. Kadyan, "Image Fusion Techniques: A Survey," Archives of Computational Methods in Engineering, vol. 28, no. 7, pp. 4425–4447, Dec. 2021. DOI: https://doi.org/10.1007/s11831-021-09540-7
P.-H. Dinh, "Medical image fusion based on enhanced three-layer image decomposition and Chameleon swarm algorithm," Biomedical Signal Processing and Control, vol. 84, Jul. 2023, Art. no. 104740. DOI: https://doi.org/10.1016/j.bspc.2023.104740
M. Haribabu, V. Guruviah, and P. Yogarajah, "Recent Advancements in Multimodal Medical Image Fusion Techniques for Better Diagnosis: An Overview," Current Medical Imaging, vol. 19, no. 7, pp. 673–694, Sep. 2022. DOI: https://doi.org/10.2174/1573405618666220606161137
R. Maurya, R. Swarnkar, Y. K. Sharma, R. Mishra, and D. M. Pathak, "Medical Image Fusion." Research Square, Jun. 06, 2024. DOI: https://doi.org/10.21203/rs.3.rs-4456478/v1
S. Chinnadurai and D. Vasanthi, "Enhancement of Medical Image by Fusion Method using Fast Discrete Curvelet Transform," International Journal for Research in Applied Science and Engineering Technology, vol. 7, no. 5, pp. 344–350, May 2019. DOI: https://doi.org/10.22214/ijraset.2019.5057
P. Kavita, D. R. Alli, and A. B. Rao, "Study of image fusion optimization techniques for medical applications," International Journal of Cognitive Computing in Engineering, vol. 3, pp. 136–143, Jun. 2022. DOI: https://doi.org/10.1016/j.ijcce.2022.05.002
S. R. K. Reddy, V. Swathi, and K. Anusha, "MR and CT Image Fusion Using Nonlinear Anisotropic Filtering in PCA Domain," Journal of Physics: Conference Series, vol. 1964, no. 6, Jul. 2021, Art. no. 062058. DOI: https://doi.org/10.1088/1742-6596/1964/6/062058
M. G. Reddy, P. V. N. Reddy, and P. R. Reddy, "Medical Image Fusion Using Integrated Guided Nonlinear Anisotropic Filtering with Image Statistics," International Journal of Intelligent Engineering and Systems, vol. 13, no. 1, pp. 25–34, Feb. 2020. DOI: https://doi.org/10.22266/ijies2020.0229.03
F. Shabanzade and H. Ghassemian, "Combination of wavelet and contourlet transforms for PET and MRI image fusion," in 2017 Artificial Intelligence and Signal Processing Conference, Shiraz, Iran, 2017, pp. 178–183. DOI: https://doi.org/10.1109/AISP.2017.8324077
A. Kesharwani, K. Singh, and A. Saxena, "Advancements In Multi-Modality Medical Image Fusion: A Comprehensive Review," International Journal of Innovative Research in Engineering and Management, vol. 11, no. 2, pp. 55–61, Apr. 2024. DOI: https://doi.org/10.55524/ijirem.2024.11.2.11
T. Zhou, Q. Cheng, H. Lu, Q. Li, X. Zhang, and S. Qiu, "Deep learning methods for medical image fusion: A review," Computers in Biology and Medicine, vol. 160, Jun. 2023, Art. no. 106959. DOI: https://doi.org/10.1016/j.compbiomed.2023.106959
Y. Zhou, X. Yang, S. Liu, and J. Yin, "Multimodal Medical Image Fusion Network Based on Target Information Enhancement," IEEE Access, vol. 12, pp. 70851–70869, 2024. DOI: https://doi.org/10.1109/ACCESS.2024.3402965
W. Xu, Y.-L. Fu, H. Xu, and K. K. L. Wong, "Medical image fusion using enhanced cross-visual cortex model based on artificial selection and impulse-coupled neural network," Computer Methods and Programs in Biomedicine, vol. 229, Feb. 2023, Art. no. 107304. DOI: https://doi.org/10.1016/j.cmpb.2022.107304
H. Chen, L. Jiao, M. Liang, F. Liu, S. Yang, and B. Hou, "Fast unsupervised deep fusion network for change detection of multitemporal SAR images," Neurocomputing, vol. 332, pp. 56–70, Mar. 2019. DOI: https://doi.org/10.1016/j.neucom.2018.11.077
B. Zhan, D. Li, X. Wu, J. Zhou, and Y. Wang, "Multi-Modal MRI Image Synthesis via GAN With Multi-Scale Gate Mergence," IEEE Journal of Biomedical and Health Informatics, vol. 26, no. 1, pp. 17–26, Jan. 2022. DOI: https://doi.org/10.1109/JBHI.2021.3088866
J. Huang, Z. Le, Y. Ma, F. Fan, H. Zhang, and L. Yang, "MGMDcGAN: Medical Image Fusion Using Multi-Generator Multi-Discriminator Conditional Generative Adversarial Network," IEEE Access, vol. 8, pp. 55145–55157, 2020. DOI: https://doi.org/10.1109/ACCESS.2020.2982016
L. Wang, C. Chang, B. Hao, and C. Liu, "Multi-modal Medical Image Fusion Based on GAN and the Shift-Invariant Shearlet Transform," in 2020 IEEE International Conference on Bioinformatics and Biomedicine, Seoul, South Korea, 2020, pp. 2538–2543. DOI: https://doi.org/10.1109/BIBM49941.2020.9313288
J. Kang, W. Lu, and W. Zhang, "Fusion of Brain PET and MRI Images Using Tissue-Aware Conditional Generative Adversarial Network With Joint Loss," IEEE Access, vol. 8, pp. 6368–6378, 2020. DOI: https://doi.org/10.1109/ACCESS.2019.2963741
K. S. Prasad, M. Kolli, B. Linga, S. S. Chikati, and T. Veeranki, "Enhancing Medical Diagnosis Through Multimodal Medical Image Fusion," in Enhancing Medical Imaging with Emerging Technologies, A. K. Sharma, N. Chanderwal, S. Tyagi, P. Upadhyay, and A. K. Tyagi, Eds. Hershey, PA, USA: IGI Global Scientific Publishing, 2024, pp. 197–209. DOI: https://doi.org/10.4018/979-8-3693-5261-8.ch012
M. A. Azam et al., "A review on multimodal medical image fusion: Compendious analysis of medical modalities, multimodal databases, fusion techniques and quality metrics," Computers in Biology and Medicine, vol. 144, May 2022, Art. no. 105253. DOI: https://doi.org/10.1016/j.compbiomed.2022.105253
J. Jose et al., "An image quality enhancement scheme employing adolescent identity search algorithm in the NSST domain for multimodal medical image fusion," Biomedical Signal Processing and Control, vol. 66, Apr. 2021, Art. no. 102480. DOI: https://doi.org/10.1016/j.bspc.2021.102480
V. Ramaraj, M. Venkatachalaappaswamy, and M. K. Sankar, "Medical Image Fusion for Brain Tumor Diagnosis Using Effective Discrete Wavelet Transform Methods," Journal of Information Systems Engineering and Business Intelligence, vol. 10, no. 1, pp. 70–80, Feb. 2024. DOI: https://doi.org/10.20473/jisebi.10.1.70-80
I. Q. Abduljaleel and I. H. Ali, "Deep Learning and Fusion Mechanism-based Multimodal Fake News Detection Methodologies: A Review," Engineering, Technology & Applied Science Research, vol. 14, no. 4, pp. 15665–15675, Aug. 2024. DOI: https://doi.org/10.48084/etasr.7907
H. G. Doan and N. T. Nguyen, "Fusion Machine Learning Strategies for Multi-modal Sensor-based Hand Gesture Recognition," Engineering, Technology & Applied Science Research, vol. 12, no. 3, pp. 8628–8633, Jun. 2022. DOI: https://doi.org/10.48084/etasr.4913
D.-H. Xia et al., "Review-material degradation assessed by digital image processing: Fundamentals, progresses, and challenges," Journal of Materials Science & Technology, vol. 53, pp. 146–162, Sep. 2020. DOI: https://doi.org/10.1016/j.jmst.2020.04.033
B. H. Menze et al., "The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)," IEEE Transactions on Medical Imaging, vol. 34, no. 10, pp. 1993–2024, Oct. 2015.
K. A. Johnson and J. A. Becker. "The Whole Brain Atlas." Harvard Medical School. https://www.med.harvard.edu/AANLIB/.
W. Huang, H. Zhang, H. Guo, W. Li, X. Quan, and Y. Zhang, "ADDNS: An asymmetric dual deep network with sharing mechanism for medical image fusion of CT and MR-T2," Computers in Biology and Medicine, vol. 166, Nov. 2023, Art. no. 107531. DOI: https://doi.org/10.1016/j.compbiomed.2023.107531
Downloads
How to Cite
License
Copyright (c) 2025 T. M. Hayath, Sai Madhavi D
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
