Re-thinking large scale hate speech identification: beyond common NLP conventions and supervised machine learning

Abstract

The detection of hate speech in online spaces is traditionally conceptualized as a classification task that uses Machine Learning (ML)-driven Natural Language Processing (NLP) techniques. In accordance with this conceptualization, the hate speech detection task relies upon common conventions and practices in Artificial Intelligence, ML and NLP – among them interpretation of the inter-annotator agreement as a way to measure dataset quality and the use of standard metrics such as precision, recall or accuracy and benchmarks to assess model performance. However, hate speech is a highly subjective and context-dependent notion that eludes such static and disembodied practices. Their application results in definitorial challenges and the failure of the models to generalize across different datasets, two problems that I analyse in empirical studies. Furthermore, I critically reflect on the followed methodologies. I argue that many conventions in NLP are poorly suited for the problem and suggest to develop methods that are more appropriate for fighting online hate speech.