USHAHEMBA, TERHEMBAFAN HILARY (2026) AI BASED MULTIMODAL MENTAL HEALTH MONITORING SYSTEM. Other thesis, Godfrey Okoye University, Enugu.
|
Text
TERHEMBAFAN HILARY USHAHEMBA PROJECT 910 (Autosaved).pdf Download (2MB) | Preview |
Abstract
Mental health problems such as stress, anxiety, depression, and emotional trauma impact people all around the world; nonetheless, timely emotional assessments are still a concern to people worldwide, especially due to limitations in access to mental health care. Current digital-based mental health monitoring systems mostly rely on only one emotional assessment, providing an incomplete view of the person's emotional state. Here, we describe the design and implementation of an AI-Based Multimodal Mental Health Monitoring System that analyses emotional information from texts, user's facial emotions, and user's screens behavior to give a more complete view of the person’s emotional state. For textual emotion analysis, the fine-tuned DistilBERT has been applied, while for face emotion detection, MobileNetV2 was applied. The behavioural analysis, on the other hand, has been designed with a rule-based system that analyses user-system interactions based on, for example, the time spent on the device, the frequency of actions performed, the pattern of usage of the device and user's inactivity (idle). Those analyses are fused together in order to give us some emotional insights and provide recommendations to improve user’s wellbeing. The system has been developed in Python with FastAPI, React, Javascript, PostgreSQL, and PyTorch and Tensorflow for machine learning purposes. For text emotion recognition we have used a text dataset composed of 5,937 textual samples of three emotions (Anger, Fear, Joy), while for facial expression detection we have used a dataset of 4,948 facial samples categorized as Angry, Happy, and Sad. Data cleaning, transfer learning, hyperparameters tuning and testing have been conducted in order to optimise the performances of the models. With fine-tuned DistilBERT models achieved accuracy up to 99.92%, while optimized MobileNetV2 was able to reach up to 89.00%. The experiment results showcase the capability of transfer learning to adapt already trained models for the tasks described above and prove the effectiveness of the integrated approach for the development of an artificial intelligence based system that uses multiple streams of information in order to detect a person’s emotional state, to understand emotional fluctuations, and to provide personalized assistance. Conclusively, this work points to the great potential of using artificial intelligence with multiple streams of data for improving people emotional assessments, leading towards more aware, responsible and timely digital-based mental health care, interventions, and support.
| Item Type: | Thesis (Other) |
|---|---|
| Subjects: | Q Science > Q Science (General) |
| Divisions: | Faculty of Engineering, Science and Mathematics > School of Electronics and Computer Science |
| Depositing User: | MICHAEL MADUBUKO |
| Date Deposited: | 22 Jul 2026 10:29 |
| Last Modified: | 22 Jul 2026 10:29 |
| URI: | http://eprints.gouni.edu.ng/id/eprint/5981 |
Actions (login required)
 |
View Item |
