Analyzing Hybrid Feature Representations for Improved Multiclass Bug Severity Classification
Received: 22 March 2025 | Revised: 16 May 2025 and 18 May 2025 | Accepted: 21 May 2025 | Online: 2 August 2025
Corresponding author: Jantima Polpinij
Abstract
The severity classification of software bugs plays a crucial role in the field of software maintenance, as it enables developers to prioritize issues. The present study investigates the effectiveness of hybrid feature representations in improving multiclass bug severity classification using Mozilla bug reports, which contain five predefined severity levels. This study explores the impact of integrating traditional statistical features—Term Frequency-Inverse Document Frequency (TF-IDF)—with contextual word embeddings, including Word2Vec, FastText, and Bidirectional Encoder Representations from Transformers (BERT), to enhance classification performance. Additionally, an assessment is conducted to determine the influence of feature selection techniques, including no selection (all features), Least Absolute Shrinkage and Selection Operator (LASSO), and Principal Component Analysis (PCA), on model performance and training efficiency. The classification performance is measured using three machine learning models: Logistic Regression (LR), Support Vector Machine (SVM), and Random Forest (RF). The results demonstrate that incorporating word embeddings with TF-IDF consistently improves the performance of LR across all cases, achieving an accuracy range of 65.95%–66.15%, compared to 65.69% with TF-IDF alone. Furthermore, applying LASSO for feature selection has been shown to significantly reduce training time while enhancing the performance of SVM. However, the efficacy of hybrid feature representations was found to be less effective for RF. These findings highlight the benefits of hybrid feature representations and feature selection techniques in text-based multiclass classification tasks. This research provides valuable insights for optimizing bug severity classification and can be extended to other domains requiring effective text classification strategies.
Keywords:
bug severity classification, hybrid feature representation, word embeddings, feature selection, machine learningDownloads
References
C. Vassallo, G. Grano, F. Palomba, H. C. Gall, and A. Bacchelli, "A large-scale empirical exploration on refactoring activities in open source software projects," Science of Computer Programming, vol. 180, pp. 1–15, Jul. 2019. DOI: https://doi.org/10.1016/j.scico.2019.05.002
S. Akbarinasaji, B. Caglayan, and A. Bener, "Predicting bug-fixing time: A replication study using an open source software project," Journal of Systems and Software, vol. 136, pp. 173–186, Feb. 2018. DOI: https://doi.org/10.1016/j.jss.2017.02.021
N. Serrano and I. Ciordia, "Bugzilla, ITracker, and other bug trackers," IEEE Software, vol. 22, no. 2, pp. 11–13, Mar. 2005. DOI: https://doi.org/10.1109/MS.2005.32
H. M. Tran, S. T. Le, S. V. Nguyen, and P. T. Ho, "An Analysis of Software Bug Reports Using Machine Learning Techniques," SN Computer Science, vol. 1, no. 1, Jun. 2019, Art. no. 4. DOI: https://doi.org/10.1007/s42979-019-0004-1
D.-G. Lee and Y.-S. Seo, "Improving bug report triage performance using artificial intelligence based document generation model," Human-centric Computing and Information Sciences, vol. 10, no. 1, Jun. 2020, Art. no. 26. DOI: https://doi.org/10.1186/s13673-020-00229-7
T. S. S. Angel, G. S. Kumar, V. M. Sehgal, and G. Nayak, "Effective Bug Processing and Tracking System," Journal of Computational and Theoretical Nanoscience, vol. 15, no. 8, pp. 2604–2606, Aug. 2018. DOI: https://doi.org/10.1166/jctn.2018.7506
S. Fang, Y. Tan, T. Zhang, Z. Xu, and H. Liu, "Effective Prediction of Bug-Fixing Priority via Weighted Graph Convolutional Networks," IEEE Transactions on Reliability, vol. 70, no. 2, pp. 563–574, Jun. 2021. DOI: https://doi.org/10.1109/TR.2021.3074412
W. Y. Ramay, Q. Umer, X. C. Yin, C. Zhu, and I. Illahi, "Deep Neural Network-Based Severity Prediction of Bug Reports," IEEE Access, vol. 7, pp. 46846–46857, 2019. DOI: https://doi.org/10.1109/ACCESS.2019.2909746
A. Lamkanfi, S. Demeyer, E. Giger, and B. Goethals, "Predicting the severity of a reported bug," in 2010 7th IEEE Working Conference on Mining Software Repositories, Cape Town, South Africa, 2010, pp. 1–10. DOI: https://doi.org/10.1109/MSR.2010.5463284
K. Sarawan, J. Polpinij, and B. Luaphol, "Machine Learning-Based Methods for Identifying Bug Severity Level from Bug Reports," in Proceedings of the 19th International Conference on Computing and Information Technology, Bangkok, Thailand, 2023, pp. 199–208. DOI: https://doi.org/10.1007/978-3-031-30474-3_17
N. K.-S. Roy and B. Rossi, "Towards an Improvement of Bug Severity Classification," in 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications, Verona, Italy, 2014, pp. 269–276.
A.-H. Dao and C.-Z. Yang, "Severity Prediction for Bug Reports Using Multi-Aspect Features: A Deep Learning Approach," Mathematics, vol. 9, no. 14, Jul. 2021, Art. no. 1644. DOI: https://doi.org/10.3390/math9141644
S. Fang, T. Zhang, Y. Tan, H. Jiang, X. Xia, and X. Sun, "RepresentThemAll: A Universal Learning Representation of Bug Reports," in 2023 IEEE/ACM 45th International Conference on Software Engineering, Melbourne, Australia, 2023, pp. 602–614. DOI: https://doi.org/10.1109/ICSE48619.2023.00060
Y. Wei, C. Zhang, and T. Ren, "Improving Bug Severity Prediction With Domain-Specific Representation Learning," IEEE Access, vol. 11, pp. 62829–62839, 2023. DOI: https://doi.org/10.1109/ACCESS.2023.3279205
J. Kim and G. Yang, "Bug Severity Prediction Algorithm Using Topic-Based Feature Selection and CNN-LSTM Algorithm," IEEE Access, vol. 10, pp. 94643–94651, 2022. DOI: https://doi.org/10.1109/ACCESS.2022.3204689
A. Kukkar, R. Mohana, and Y. Kumar, "Does bug report summarization help in enhancing the accuracy of bug severity classification?," Procedia Computer Science, vol. 167, pp. 1345–1353, 2020. DOI: https://doi.org/10.1016/j.procs.2020.03.345
Y. Tan, S. Xu, Z. Wang, T. Zhang, Z. Xu, and X. Luo, "Bug severity prediction using question-and-answer pairs from Stack Overflow," Journal of Systems and Software, vol. 165, Jul. 2020, Art. no. 110567. DOI: https://doi.org/10.1016/j.jss.2020.110567
S. Sharmin, F. Aktar, A. A. Ali, M. A. H. Khan, and M. Shoyaib, "BFSp: A feature selection method for bug severity classification," in 2017 IEEE Region 10 Humanitarian Technology Conference, Dhaka, Bangladesh, 2017, pp. 750–754. DOI: https://doi.org/10.1109/R10-HTC.2017.8289066
L. A. F. Gomes, R. da S. Torres, and M. L. Côrtes, "Bug report severity level prediction in open source software: A survey and research opportunities," Information and Software Technology, vol. 115, pp. 58–78, Nov. 2019. DOI: https://doi.org/10.1016/j.infsof.2019.07.009
T. Zhang, J. Chen, G. Yang, B. Lee, and X. Luo, "Towards more accurate severity prediction and fixer recommendation of software bugs," Journal of Systems and Software, vol. 117, pp. 166–184, Jul. 2016. DOI: https://doi.org/10.1016/j.jss.2016.02.034
A. Lamkanfi, J. Pérez, and S. Demeyer, "The Eclipse and Mozilla defect tracking dataset: A genuine dataset for mining bug information," in 2013 10th Working Conference on Mining Software Repositories, San Francisco, CA, USA, 2013, pp. 203–206. DOI: https://doi.org/10.1109/MSR.2013.6624028
K. Sparck Jones, "A Statistical Interpretation of Term Specificity and its Application in Retrieval," Journal of Documentation, vol. 28, no. 1, pp. 11–21, Jan. 1972. DOI: https://doi.org/10.1108/eb026526
D. Ali, M. M. S. Missen, and M. Husnain, "Multiclass Event Classification from Text," Scientific Programming, vol. 2021, no. 1, Jan. 2021, Art. no. 6660651. DOI: https://doi.org/10.1155/2021/6660651
W. Zhang, T. Yoshida, and X. Tang, "A comparative study of TF*IDF, LSI and multi-words for text classification," Expert Systems with Applications, vol. 38, no. 3, pp. 2758–2765, Mar. 2011. DOI: https://doi.org/10.1016/j.eswa.2010.08.066
D. S. Asudani, N. K. Nagwani, and P. Singh, "Impact of word embedding models on text analytics in deep learning environment: a review," Artificial Intelligence Review, vol. 56, no. 9, pp. 10345–10425, Sep. 2023. DOI: https://doi.org/10.1007/s10462-023-10419-1
S. Selva Birunda and R. Kanniga Devi, "A Review on Word Embedding Techniques for Text Classification," in Innovative Data Communication Technologies and Application: Proceedings of ICIDCA 2020, Coimbatore, India, 2021, pp. 267–281. DOI: https://doi.org/10.1007/978-981-15-9651-3_23
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA, 2019, pp. 4171–4186.
A. K. Chanda, "Efficacy of BERT embeddings on predicting disaster from Twitter data." arXiv, Aug. 08, 2021.
P. Bojanowski, E. Grave, A. Joulin, and T. Mikolov, "Enriching Word Vectors with Subword Information," Transactions of the Association for Computational Linguistics, vol. 5, pp. 135–146, Jun. 2017. DOI: https://doi.org/10.1162/tacl_a_00051
B. Athiwaratkun, A. G. Wilson, and A. Anandkumar, "Probabilistic FastText for Multi-Sense Word Embeddings." arXiv, Jun. 07, 2018. DOI: https://doi.org/10.18653/v1/P18-1001
G. Di Gennaro, A. Buonanno, and F. A. N. Palmieri, "Considerations about learning Word2Vec," The Journal of Supercomputing, vol. 77, no. 11, pp. 12320–12335, Nov. 2021. DOI: https://doi.org/10.1007/s11227-021-03743-2
S. J. Johnson, M. R. Murty, and I. Navakanth, "A detailed review on word embedding techniques with emphasis on word2vec," Multimedia Tools and Applications, vol. 83, no. 13, pp. 37979–38007, Apr. 2024. DOI: https://doi.org/10.1007/s11042-023-17007-z
Y. Tang, H. Zhou, and H. Su, "Automatic Classification of Software Bug Reports Based on LDA and Word2Vec," in 2022 2nd International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology, Nanjing, China, 2022, pp. 491–495. DOI: https://doi.org/10.1109/CEI57409.2022.9950207
K. Wang, L. Liu, C. Yuan, and Z. Wang, "Software defect prediction model based on LASSO–SVM," Neural Computing and Applications, vol. 33, no. 14, pp. 8249–8259, Jul. 2021. DOI: https://doi.org/10.1007/s00521-020-04960-1
R. Tibshirani, "Regression Shrinkage and Selection Via the Lasso," Journal of the Royal Statistical Society: Series B (Methodological), vol. 58, no. 1, pp. 267–288, Jan. 1996. DOI: https://doi.org/10.1111/j.2517-6161.1996.tb02080.x
S. Iqbal, R. Naseem, S. Jan, S. Alshmrany, M. Yasar, and A. Ali, "Determining Bug Prioritization Using Feature Reduction and Clustering With Classification," IEEE Access, vol. 8, pp. 215661–215678, 2020. DOI: https://doi.org/10.1109/ACCESS.2020.3035063
A. Maćkiewicz and W. Ratajczak, "Principal components analysis (PCA)," Computers & Geosciences, vol. 19, no. 3, pp. 303–342, Mar. 1993. DOI: https://doi.org/10.1016/0098-3004(93)90090-R
Z. H. Arif and K. Cengiz, "Severity Classification for COVID-19 Infections based on Lasso-Logistic Regression Model," International Journal of Mathematics, Statistics, and Computer Science, vol. 1, pp. 25–32, Apr. 2023. DOI: https://doi.org/10.59543/ijmscs.v1i.7715
H. Shi, S. Liu, J. Chen, X. Li, Q. Ma, and B. Yu, "Predicting drug-target interactions using Lasso with random forest based on evolutionary information and chemical structure," Genomics, vol. 111, no. 6, pp. 1839–1852, Dec. 2019. DOI: https://doi.org/10.1016/j.ygeno.2018.12.007
S. R. Basha, J. K. Rani, and J. J. C. P. Yadav, "A Novel Summarization-based Approach for Feature Reduction Enhancing Text Classification Accuracy," Engineering, Technology & Applied Science Research, vol. 9, no. 6, pp. 5001–5005, Dec. 2019. DOI: https://doi.org/10.48084/etasr.3173
Downloads
How to Cite
License
Copyright (c) 2025 Kamthorn Sarawan, Jantima Polpinij, Gamgarn Somprasertsri, Bancha Luaphol
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
