DEVELOPMENT OF A HYBRID MALWARE DETECTION SYSTEM USING BEHAVIORAL FEATURES

UBI, ESTHER (2026) DEVELOPMENT OF A HYBRID MALWARE DETECTION SYSTEM USING BEHAVIORAL FEATURES. Other thesis, Godfrey Okoye University, Enugu.

[img]
Preview
Text
Final_Year_Project_Doc.pdf

Download (986kB) | Preview

Abstract

Malware detection remains a critical issue in the cybersecurity domain. As the prevalence of obfuscated, polymorphic and unknown malicious attacks grow, traditional signature-based methods struggle to keep pace. This work addresses some of these shortcomings of single-model malware detection techniques and lack of interpretable explanations of their behaviors by proposing and developing a hybrid system based on behavioral analysis and stacking ensembles in machine learning. A total of 49,179 malicious Windows PE samples were gathered from the Avast-CTU CAPEv2 dataset and a collection of benign samples curated by a researcher to train the system. Forty-two static PE and behavioral features were extracted from sandbox execution reports utilizing a custom feature extraction mechanism. Two complementory base learners, an XGBoost gradient boosting classifier and a PyTorch Multilayer Perceptron (MLP) were used and combined using a stacking ensemble architecture with Logistic Regression as the meta-learner. Models were trained and tested on identically same settings. A deployment threshold was fixed at 0.96. The results confirmed that the proposed hybrid ensemble model outperforms its base learners and reached the accuracy of precision, recall and F1-score of 1.0000, 0.9999 and 0.9999 with the false positive rate 0.0000 for a test data size of 9,836 samples. Alongside the high detection performance, the system also provides rule-based interpretable explanations of the detected malware by translating model prediction into simple human understandable behavioral observations. The obtained system was implemented as a web-based application incorporating a FastAPI backend, CAPE sandbox, and React frontend. The proposed work successfully shows that a hybrid stacking ensemble model can enhance the detection performance of malware detection system while maintaining interpretability.

Item Type: Thesis (Other)
Subjects: Q Science > Q Science (General)
Divisions: Faculty of Engineering, Science and Mathematics > School of Electronics and Computer Science
Depositing User: MICHAEL MADUBUKO
Date Deposited: 22 Jul 2026 13:46
Last Modified: 22 Jul 2026 13:46
URI: http://eprints.gouni.edu.ng/id/eprint/6031

Actions (login required)

View Item View Item