Multimodal Hate Event Detection in Russia-Ukraine War

Paper ID: 34

Abstract

This study introduces a unified multimodal fusion model for hate speech detection (HD) and target detection (TD) using the CrisisHateMM dataset of 4,700 annotated text-embedded images. Combining TwHIN-BERT for text and Vision Transformer (ViT) for visuals, the model achieved state-of-the-art F1-scores of 0.806 (HD) and 0.683 (TD). Future work focuses on enhancing model performance and expanding datasets to strengthen online hate speech mitigation during crises.

Keywords