Multi-Class Brain Tumor Diagnosis Using a Vision Transformer with MRI Image Segmentation
Received: 20 April 2025 | Revised: 5 June 2025 | Accepted: 14 June 2025 | Online: 2 August 2025
Corresponding author: Kunjam Nageswara Rao
Abstract
Over the last decade, brain tumors have emerged as a significant and potentially fatal medical issue. Traditional methods for detecting and classifying brain tumors using Magnetic Resonance Imaging (MRI) scans are often time-consuming and prone to inaccuracies, necessitating a precise classification method for brain tumors, effective diagnosis, and therapy planning. This study proposes the use of a Vision Transformer (ViT) model to classify brain tumors from the Brain Tumor MRI dataset available on Kaggle into four categories: no-tumor, meningioma, pituitary tumor, and glioma. Unlike conventional Convolutional Neural Networks (CNNs), the proposed ViT model leverages self-attention mechanisms, making it particularly effective for capturing global relationships and extracting complex features from medical images. The model processes images by dividing them into fixed-size patches, which are then linearly embedded and passed through a positional encoding layer. These encoded representations are input into the transformer’s encoding layers, and the final classification is produced through a fully connected output layer. The performance of the ViT model is evaluated using standard multi-class classification metrics. The model achieved an impressive average accuracy of 99.3%, outperforming all other Deep Learning (DL) models previously tested on this benchmark dataset. The ViT model’s auto-focusing capability enables it to capture both fine-grained and large-scale features, significantly improving the accuracy and reliability of brain tumor detection.
Keywords:
brain tumor, Vision Transformers (ViT), Magnetic Resonance Imaging (MRI) images, multi-class classification, self-attention, CNNReferences
D. N. Louis et al., "The 2021 WHO Classification of Tumors of the Central Nervous System: a summary," Neuro-Oncology, vol. 23, no. 8, pp. 1231–1251, Aug. 2021. DOI: https://doi.org/10.1093/neuonc/noab106
M. L. Bondy et al., "Brain tumor epidemiology: Consensus from the Brain Tumor Epidemiology Consortium," Cancer, vol. 113, no. S7, pp. 1953–1968, Oct. 2008. DOI: https://doi.org/10.1002/cncr.23741
A. Akter et al., "Robust clinical applicable CNN and U-Net based algorithm for MRI classification and segmentation for brain tumor," Expert Systems with Applications, vol. 238, Mar. 2024, Art. no. 122347. DOI: https://doi.org/10.1016/j.eswa.2023.122347
L. M. Deangelis, "Brain Tumors," New England Journal of Medicine, vol. 344, no. 2, pp. 114–123, Jan. 2001. DOI: https://doi.org/10.1056/NEJM200101113440207
A. Vienne-Jumeau, C. Tafani, and D. Ricard, "Environmental risk factors of primary brain tumors: A review," Revue Neurologique, vol. 175, no. 10, pp. 664–678, Dec. 2019. DOI: https://doi.org/10.1016/j.neurol.2019.08.004
M. Chintagumpala and A. Gajjar, "Brain Tumors," Pediatric Clinics of North America, vol. 62, no. 1, pp. 167–178, Feb. 2015. DOI: https://doi.org/10.1016/j.pcl.2014.09.011
M. J. Strong and J. Garces, "Brain Tumors: Epidemiology and Current Trends in Treatment," Journal of Brain Tumors & Neurooncology, vol. 01, no. 01, 2016. DOI: https://doi.org/10.4172/2475-3203.1000102
G. Kontogeorgos, "Classification and Pathology of Pituitary Tumors," Endocrine, vol. 28, no. 1, pp. 027–036, 2005. DOI: https://doi.org/10.1385/ENDO:28:1:027
K. A. Rajasekaran and C. C. Gounder, "Advanced Brain Tumour Segmentation from MRI Images," in High-Resolution Neuroimaging - Basic Physical Principles and Clinical Applications, InTech, 2018. DOI: https://doi.org/10.5772/intechopen.71416
J. E. Villanueva-Meyer, M. C. Mabray, and S. Cha, "Current Clinical Brain Tumor Imaging," Neurosurgery, vol. 81, no. 3, pp. 397–415, Sep. 2017. DOI: https://doi.org/10.1093/neuros/nyx103
C. Watson, M. Kirkcaldie, and G. Paxinos, The brain: an introduction to functional neuroanatomy, 1st ed. Amsterdam Boston: Elsevier/Academic, 2010.
M. Nazir, S. Shakil, and K. Khurshid, "Role of deep learning in brain tumor detection and classification (2015 to 2020): A review," Computerized Medical Imaging and Graphics, vol. 91, Jul. 2021, Art. no. 101940. DOI: https://doi.org/10.1016/j.compmedimag.2021.101940
J. Kang, Z. Ullah, and J. Gwak, "MRI-Based Brain Tumor Classification Using Ensemble of Deep Features and Machine Learning Classifiers," Sensors, vol. 21, no. 6, Mar. 2021, Art. no. 2222. DOI: https://doi.org/10.3390/s21062222
W. Ayadi, W. Elhamzi, I. Charfi, and M. Atri, "Deep CNN for Brain Tumor Classification," Neural Processing Letters, vol. 53, no. 1, pp. 671–700, Feb. 2021. DOI: https://doi.org/10.1007/s11063-020-10398-2
B. A.Mohammed and M. S. Al-Ani, "An efficient approach to diagnose brain tumors through deep CNN," Mathematical Biosciences and Engineering, vol. 18, no. 1, pp. 851–867, 2021. DOI: https://doi.org/10.3934/mbe.2021045
P. Saxena, A. Maheshwari, and S. Maheshwari, "Predictive modeling of brain tumor: a deep learning approach," in International Conference on Innovations in Computational Intelligence and Computer Vision (ICICV-2020), Jaipur, India, Jan. 2020. DOI: https://doi.org/10.1007/978-981-15-6067-5_30
J. T. Senders et al., "Natural Language Processing for Automated Quantification of Brain Metastases Reported in Free-Text Radiology Reports," JCO Clinical Cancer Informatics, no. 3, pp. 1–9, Dec. 2019. DOI: https://doi.org/10.1200/CCI.18.00138
A. A. Asiri et al., "Exploring the Power of Deep Learning: Fine-Tuned Vision Transformer for Accurate and Efficient Brain Tumor Detection in MRI Scans," Diagnostics, vol. 13, no. 12, Jun. 2023, Art. no. 2094. DOI: https://doi.org/10.3390/diagnostics13122094
G. Cinarer and B. G. Emiroglu, "Classification of Brain Tumors by Machine Learning Algorithms," in 2019 3rd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), Ankara, Turkey, Oct. 2019, pp. 1–4. DOI: https://doi.org/10.1109/ISMSIT.2019.8932878
S. Dua, V. D. Chakravarthy, and I. Sharma, "Binary classification of brain tumor using machine learning algorithms," in AIP Conference Proceedings, Kattankalathur, India, 2024, vol. 3075, Art. no. 020181. DOI: https://doi.org/10.1063/5.0219073
H. Ali Khan et al., "Brain tumor classification in MRI image using convolutional neural network," Mathematical Biosciences and Engineering, vol. 17, no. 5, pp. 6203–6216, 2020. DOI: https://doi.org/10.3934/mbe.2020328
D. R. Nayak, N. Padhy, P. K. Mallick, and A. Singh, "A deep autoencoder approach for detection of brain tumor images," Computers and Electrical Engineering, vol. 102, Sep. 2022, Art. no. 108238. DOI: https://doi.org/10.1016/j.compeleceng.2022.108238
P. S. Smitha, G. Balaarunesh, C. Sruthi Nath, and A. Sabatini S, "Classification of brain tumor using deep learning at early stage," Measurement: Sensors, vol. 35, Oct. 2024, Art. no. 101295, https://doi.org/10.1016/j.measen.2024.101295. DOI: https://doi.org/10.1016/j.measen.2024.101295
S. Pereira, R. Meier, V. Alves, M. Reyes, and C. A. Silva, "Automatic brain tumor grading from MRI data using convolutional neural networks and quality assessment," arXiv, 2018. DOI: https://doi.org/10.1007/978-3-030-02628-8_12
M. A. Khan et al., "Brain tumor detection and classification: A framework of marker‐based watershed algorithm and multilevel priority features selection," Microscopy Research and Technique, vol. 82, no. 6, pp. 909–922, Jun. 2019. DOI: https://doi.org/10.1002/jemt.23238
M. Sajjad, S. Khan, K. Muhammad, W. Wu, A. Ullah, and S. W. Baik, "Multi-grade brain tumor classification using deep CNN with extensive data augmentation," Journal of Computational Science, vol. 30, pp. 174–182, Jan. 2019. DOI: https://doi.org/10.1016/j.jocs.2018.12.003
H. Mzoughi et al., "Deep Multi-Scale 3D Convolutional Neural Network (CNN) for MRI Gliomas Brain Tumor Classification," Journal of Digital Imaging, vol. 33, no. 4, pp. 903–915, Aug. 2020. DOI: https://doi.org/10.1007/s10278-020-00347-9
A. Hasan, F. Meziane, R. Aspin, and H. Jalab, "Segmentation of Brain Tumors in MRI Images Using Three-Dimensional Active Contour without Edge," Symmetry, vol. 8, no. 11, Nov. 2016, Art. no. 132. DOI: https://doi.org/10.3390/sym8110132
A. Çinar and M. Yildirim, "Detection of tumors on brain MRI images using the hybrid convolutional neural network architecture," Medical Hypotheses, vol. 139, Jun. 2020, Art. no. 109684. DOI: https://doi.org/10.1016/j.mehy.2020.109684
S. Khawaldeh, U. Pervaiz, A. Rafiq, and R. Alkhawaldeh, "Noninvasive Grading of Glioma Tumor Using Magnetic Resonance Imaging with Convolutional Neural Networks," Applied Sciences, vol. 8, no. 1, Dec. 2017, Art. no. 27. DOI: https://doi.org/10.3390/app8010027
Y. Yang et al., "Glioma Grading on Conventional MR Images: A Deep Learning Study With Transfer Learning," Frontiers in Neuroscience, vol. 12, Nov. 2018. DOI: https://doi.org/10.3389/fnins.2018.00804
E. I. Papageorgiou et al., "Brain tumor characterization using the soft computing technique of fuzzy cognitive maps," Applied Soft Computing, vol. 8, no. 1, pp. 820–828, Jan. 2008. DOI: https://doi.org/10.1016/j.asoc.2007.06.006
A. K. Anaraki, M. Ayati, and F. Kazemi, "Magnetic resonance imaging-based brain tumor grades classification and grading via convolutional neural networks and genetic algorithms," Biocybernetics and Biomedical Engineering, vol. 39, no. 1, pp. 63–74, Jan. 2019. DOI: https://doi.org/10.1016/j.bbe.2018.10.004
S. Abbasi and F. Tajeripour, "Detection of brain tumor in 3D MRI images using local binary patterns and histogram orientation gradient," Neurocomputing, vol. 219, pp. 526–535, Jan. 2017. DOI: https://doi.org/10.1016/j.neucom.2016.09.051
N. Patel, V. K. Jain, A. K. Yadav, S. Bano, and D. S. Rajpoot, "Brain Tumor Detection from MRI Images Using Convolutional Neural Networks," in Proceedings of the 2024 Sixteenth International Conference on Contemporary Computing, Noida, India, Aug. 2024, pp. 114–121. DOI: https://doi.org/10.1145/3675888.3676039
M. I. Mahmud, M. Mamun, and A. Abdelgawad, "A Deep Analysis of Brain Tumor Detection from MR Images Using Deep Learning Networks," Algorithms, vol. 16, no. 4, Mar. 2023, Art. no. 176. DOI: https://doi.org/10.3390/a16040176
S. Saeedi, S. Rezayi, H. Keshavarz, and S. R. Niakan Kalhori, "MRI-based brain tumor detection using convolutional deep learning methods and chosen machine learning techniques," BMC Medical Informatics and Decision Making, vol. 23, no. 1, Jan. 2023. DOI: https://doi.org/10.1186/s12911-023-02114-6
A. Tariq, M. M. Iqbal, S. Bibi, M. H. Butt, and S. Ramzan, "Transforming Brain Tumor Diagnosis: Vision Transformers Combined with Ensemble Techniques," Journal of Population Therapeutics & Clinical Pharmacology, pp. 1072–1084, Jul. 2024. DOI: https://doi.org/10.53555/jptcp.v31i7.7195
Y. Pan et al., "Brain tumor grading based on Neural Networks and Convolutional Neural Networks," in 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Aug. 2015, pp. 699–702. DOI: https://doi.org/10.1109/EMBC.2015.7318458
M. Nickparvar, "Brain Tumor MRI Dataset." Kaggle.
I. Ahmad, Y. Liu, D. Javeed, and S. Ahmad, "A decision-making technique for solving order allocation problem using a genetic algorithm," IOP Conference Series: Materials Science and Engineering, vol. 853, no. 1, May 2020, Art. no. 012054. DOI: https://doi.org/10.1088/1757-899X/853/1/012054
P. Murala and K. N. Rao, "Deep Learning Approaches for Tumor Detection Using MRI Data," Journal of Theoretical and Applied Information Technology, vol. 103, no. 3, pp. 1117–1127, Feb. 2025.
J. Amin, M. Sharif, A. Haldorai, M. Yasmin, and R. S. Nayak, "Brain tumor detection and classification using machine learning: a comprehensive survey," Complex & Intelligent Systems, vol. 8, no. 4, pp. 3161–3183, Aug. 2022. DOI: https://doi.org/10.1007/s40747-021-00563-y
F. M. Refaat, M. M. Gouda, and M. Omar, "Detection and Classification of Brain Tumor Using Machine Learning Algorithms," Biomedical and Pharmacology Journal, vol. 15, no. 4, pp. 2381–2397, Dec. 2022. DOI: https://doi.org/10.13005/bpj/2576
Downloads
How to Cite
License
Copyright (c) 2025 Purnachandrarao Murala, Kunjam Nageswara Rao
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
