Beyond Classification: Exploring the Potential of NLP and Deep Learning for Real-Time Sentiment Analysis on Twitter

Abstract

Twitter has evolved into a pervasive societal force, serving as a platform for diverse expressions, including official statements, thoughts, and opinions. This study delves into the multifaceted nature of Twitter, conducting a sentiment analysis on an extensive dataset comprising 1.6 million tweets and an additional 24,000 tweets. Leveraging advanced techniques in Natural Language Processing (NLP) and employing machine and deep learning algorithms, our focus lies in refining sentiment analysis methods to identify and mitigate terrorist-related content within social media posts. The study initiates with a comprehensive data preprocessing phase, involving part-of-speech tagging, sentiment score assignment via SentiWordNet, and neutralization of domain-specific words and negations. This meticulous methodology enhances the quality of the data for subsequent analysis. Weighted sentiment scores are then calculated, categorizing tweets into positive, negative, or neutral sentiment categories. To assess the effectiveness of our approach, various machine learning and deep learning algorithms are employed, including ensemble methods such as majority voting and stacking. Results reveal that Bidirectional Recurrent Neural Networks consistently outperform other models, achieving remarkable accuracy rates of 96% and 98% on two distinct datasets. Furthermore, the study explores diverse feature extraction techniques, shedding light on their impact on model performance. The findings of this research contribute to the ongoing discourse on sentiment analysis, particularly in the context of identifying and addressing potential threats within social media.