Evaluating Locally Run Large Language Models on Toxic Meme Analysis

Toxic memes easily spread online, propagating stereotypes, hate, and other stronger or more nuanced types of malicious content. The sheer volume of memes requiring moderation calls for automated methods, but their multiple layers of meaning make them challenging to assess: in some cases, toxicity may stem from subtle wordplay, in others by visual references or evoking hateful symbols, etc. Large language models (LLMs) offer a promising tool for performing toxicity detection, since they can leverage a large amount of contextual information and analyzing content items in depth. In this paper, we investigate the suitability of locally run LLMs to perform such a task. Locally run large language models have several advantages over web-based models like OpenAI’s ChatGPT with respect to costs, reproducibility, and data safety. We evaluate the local models on the tasks of automatic meme analysis and toxic symbol identification, and compare the results with analyses of the online model ChatGPT. Our findings reveal that while local models identify only a limited number of toxic memes and symbols, their labels are often accurate (low recall, high precision). Although they do not achieve perfect performance, we believe these models can effectively support human content moderators.