Spam Email Detection using Transfer Learning of BERT Model

Abstract

Spam is the term for unsolicited and indiscriminate mass emails that are not wanted by the recipients and are often motivated by economic interests. Despite ethical concerns, many organizations persist in employing spam as a marketing tactic. Spam emails pose a significant challenge in today's digital landscape, potentially causing financial harm to businesses and annoyance to individual users. In order to address this issue, advances in natural language processing (NLP) have been applied to increase spam detection programs' accuracy. Specifically, efforts have been directed toward optimizing the performance of the already existing BERT (Bidirectional Encoder Representations from Transformers) transformer model. BERT utilizes attention mechanisms to contextualize the content of text data, enabling more effective discrimination between spam and non-spam (HAM) emails. The training of deep learning transformer models on text data through self-attention methods makes them significant. This dissertation explores the real-time classification of spam and ham emails using Google Bidirectional Encoder Representations from Transformers (BERT) base uncased models that have already been trained. The study trained several models with the goal of distinguishing between spam and ham emails using Enron datasets that were made publicly available. One of the models that was created performed well enough to classify emails with accuracy. Utilizing Enron datasets during the training phase allowed the model's hyperparameters to be adjusted for the best spam detection results. The same hyperparameters from our model were used to fine-tune the model. An F1-score in each model is at or above 0.9 when they are each using the appropriate dataset. 98% of the time was accurate overall, while the F1 score was 99%. The consequences and research results were examined. The study's findings demonstrated the effectiveness of the suggested strategy with remarkable performance metrics: 98% accuracy, 99% F1 score, 96% precision, and 99% recall or true positive rate (TPR). Furthermore, it was found that the true negative rate (TNR) was 73%, while the false positive rate (FPR) was 47%. The method's success demonstrates how well it can differentiate between emails that are spam and those that are not.