NYUAD-ComNets/Llama3.2_MultiModal_Memes_Hate_Detector

Llama3.2-11B based Hate Detection in Arabic MultiModal Memes

The rise of social media and online communication platforms has led to the spread of Arabic memes as a key form of digital expression. While these contents can be humorous and informative, they are also increasingly being used to spread offensive language and hate speech. Consequently, there is a growing demand for precise analysis of content in Arabic meme.

This work used Llama 3.2 with its vision capability to effectively identify hate content within Arabic memes. The evaluation is conducted using a dataset of Arabic memes proposed in the ArabicNLP MAHED 2025 challenge. The results underscore the capacity of Llama 3.2-11B fine-tuned with Arabic memes, to deliver the superior performance.

They achieve accuracy of 80.3% and macro F1 score of 73.3%.

The proposed solutions offer a more nuanced understanding of memes for accurate and efficient Arabic content moderation systems.

Examples of Arabic Memes from ArabicNLP MAHED 2025 challenge

Examples

import pandas as pd
import os
from unsloth import FastVisionModel
import torch
from datasets import load_dataset
from transformers import TextStreamer
from PIL import Image


import os
os.environ["TOKENIZERS_PARALLELISM"] = "false"


model_name = "NYUAD-ComNets/Llama3.2_MultiModal_Memes_Hate_Detector"

model, tokenizer = FastVisionModel.from_pretrained(model_name, token='xxxxxxxxxxxxxxxxxxxxxx')




FastVisionModel.for_inference(model)


dataset_test = load_dataset("QCRI/Prop2Hate-Meme", split = "test")


print(dataset_test)

def add_labels_column(example):
    example["labels"] = "no_hate" if example["hate_label"] == 0 else "hate"
    return example

dataset_test = dataset_test.map(add_labels_column)




pred=[]

for k in range(606):
        image = dataset_test[k]["image"]
        text = dataset_test[k]["text"]
        lab = dataset_test[k]["labels"]
        
        
        messages = [
           
            {"role": "user", "content": [
                {"type": "image"},
                {"type": "text", "text": text}
            ]}
        ]
         
        input_text = tokenizer.apply_chat_template(messages,add_generation_prompt = True)
        inputs = tokenizer(
            image,
            input_text,
            add_special_tokens = False,
            return_tensors = "pt",
        ).to("cuda")
        
        text_streamer = TextStreamer(tokenizer, skip_prompt = True)

        p = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 128,
                           use_cache = False, temperature = 0.3, min_p = 0.3)

 
        p = tokenizer.decode(p[0], skip_special_tokens=True)

        pred.append(p.split('assistant')[1].strip())

print(pred)

We used Low-Rank Adaptation (LoRA) as the Parameter-Efficient Fine-Tuning (PEFT) method for fine-tuning utilizing the unsloth framework.

The hyper-parameters of Llama 3.2-11B are as follows:

the training batch size per device is set to 4. gradients are accumulated over 4 steps. the learning rate warm-up lasts for 5 steps. the total number of training steps is 150. the learning rate is set to 0.0002. the optimizer used is 8-bit AdamW weight decay is set to 0.01. a linear learning rate scheduler is used.

BibTeX entry and citation info

@inproceedings{aldahoul2025nyuad,
  title={NYUAD at MAHED Shared Task: Detecting Hope, Hate, and Emotion in Arabic Textual Speech and Multi-modal Memes Using Large Language Models},
  author={Aldahoul, Nouar and Zaki, Yasir},
  booktitle={Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks},
  pages={575--584},
  year={2025}
}


@misc{aldahoul2025detectinghopehateemotion,
      title={Detecting Hope, Hate, and Emotion in Arabic Textual Speech and Multi-modal Memes Using Large Language Models}, 
      author={Nouar AlDahoul and Yasir Zaki},
      year={2025},
      eprint={2508.15810},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2508.15810}, 
}