Attention-Based Multi-modal Emotion Recognition from Art

Tsegaye Misikir Tashu*, Tomáš Horváth

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

2 Citations (Scopus)

Abstract

Emotions are very important in dealing with human decisions, interactions, and cognitive processes. Art is an imaginative human creation that should be appreciated, thought-provoking, and elicits an emotional response. The automatic recognition of emotions triggered by art is of considerable importance. It can be used to categorize artworks according to the emotions they evoke, recommend paintings that accentuate or balance a particular mood, and search for paintings of a particular style or genre that represent custom content in a custom state of impact. In this paper, we propose an attention-based multi-modal approach to emotion recognition that aims to use information from both the painting and title channels to achieve more accurate emotion recognition. Experimental results on the WikiArt emotion dataset showed the efficiency of the model we proposed and the usefulness of image and text modalities in emotion recognition.

Original languageEnglish
Title of host publicationPattern Recognition. ICPR International Workshops and Challenges, 2021, Proceedings
EditorsAlberto Del Bimbo, Rita Cucchiara, Stan Sclaroff, Giovanni Maria Farinella, Tao Mei, Marco Bertini, Hugo Jair Escalante, Roberto Vezzani
PublisherSpringer Science and Business Media Deutschland GmbH
Pages604-612
Number of pages9
ISBN (Print)9783030687953
DOIs
Publication statusPublished - 2021
Externally publishedYes
Event25th International Conference on Pattern Recognition Workshops, ICPR 2020 - Milan, Italy
Duration: 10-Jan-202111-Jan-2021

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12663 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference25th International Conference on Pattern Recognition Workshops, ICPR 2020
Country/TerritoryItaly
CityMilan
Period10/01/202111/01/2021

Keywords

  • Emotion analysis
  • Emotion recognition
  • Multi-modal

Fingerprint

Dive into the research topics of 'Attention-Based Multi-modal Emotion Recognition from Art'. Together they form a unique fingerprint.

Cite this