Hate Speech Detection in Twitter: Natural Language Processing Exploration

Abstract

The proliferation of social media platforms, particularly Twitter, has led to a significant rise in hate speech propagation, posing serious challenges to information dissemination and societal harmony. This paper proposes a novel approach leveraging state-of-the-art natural language processing (NLP) and deep learning techniques to automatically detect and prevent hate speech in real-time on Twitter. By employing machine learning algorithms and deep learning models such as Simple Recurrent Neural Network (SimpleRNN), Long Short-Term Memory Network (LSTM), and Gated Recurrent Unit (GRU), this study aims to surpass existing methods in hate speech classification. Utilising a dataset from Kaggle, the research conducts sentiment analysis and hate speech detection, addressing challenges such as data pre-processing and class imbalance. Various resampling techniques and model architectures are explored to optimise performance metrics including accuracy, precision, recall, F1-score, and area under the precision-recall curve (pr_auc score). The results indicated that while the Naïve Bayes algorithm achieved high precision, deep learning models, particularly best-performing LSTM architecture 2 - include accuracy: 0.950, precision: 0.633, recall: 0.674, F1-score: 0.653, pr_auc score: 0.622, and roc_auc score: 0.870, exhibited promising performance, albeit slightly below baseline expectations. Challenges such as limited training data and imbalanced datasets were identified as key factors impacting model performance. In conclusion, this research underscores the feasibility of leveraging NLP and deep learning for hate speech detection on social media platforms like Twitter. Future work entails exploring advanced models like BERT and ensemble methods to further enhance classification accuracy and mitigate the impact of data scarcity and imbalance.