ArchiMeDe@ DANKMEMES: A New Model Architecture for Meme Detection

Jinen Setpal*, Gabriele Sarti

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

We introduce ArchiMeDe, a multimodal neural network-based architecture used to solve the DANKMEMES meme detections subtask at the 2020 EVALITA campaign. The system incorporates information from visual and textual sources through a multimodal neural ensemble to predict if input images and their respective metadata are memes or not. Each pretrained neural network in the ensemble is first fine-tuned individually on the training dataset to perform domain adaptation. Learned text and visual representations are then concatenated to obtain a single multimodal embedding, and the final prediction is performed through majority voting by all networks in the ensemble.
Original languageEnglish
Title of host publicationProceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2020)
EditorsValerio Basile, Danilo Croce, Maria Di Maro, Lucia Passaro
PublisherCEUR Workshop Proceedings (CEUR-WS.org)
Publication statusPublished - 17-Dec-2020
Externally publishedYes
EventEvaluation Campaign of Natural Language Processing and Speech Tools for Italian - Online
Duration: 17-Dec-2020 → …
Conference number: 7

Workshop

WorkshopEvaluation Campaign of Natural Language Processing and Speech Tools for Italian
Abbreviated titleEVALITA 2020
Period17/12/2020 → …

Keywords

  • natural language processing
  • meme detection
  • multimodal
  • computer vision
  • italian language

Cite this