Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures

Raffaella Bernardi, Ruket Cakici, Desmond Elliott, Aykut Erdem, Erkut Erdem, Nazli Ikizler-Cinbis, Frank Keller, Adrian Muscat, Barbara Plank

    Research output: Contribution to journalArticleAcademicpeer-review

    267 Citations (Scopus)
    380 Downloads (Pure)

    Abstract

    Automatic description generation from natural images is a challenging problem that has recently received a large amount of interest from the computer vision and natural language processing communities. In this survey, we classify the existing approaches based on how they conceptualize this problem, viz., models that cast description as either generation problem or as a retrieval problem over a visual or multimodal representational space. We provide a detailed review of existing models, highlighting their advantages and disadvantages. Moreover, we give an overview of the benchmark image datasets and the evaluation measures that have been developed to assess the quality of machine-generated image descriptions. Finally we extrapolate future directions in the area of automatic image description generation.
    Original languageEnglish
    Article number4900
    Pages (from-to)409-442
    Number of pages34
    JournalJournal of artificial intelligence research
    Volume55
    DOIs
    Publication statusPublished - 2016

    Fingerprint

    Dive into the research topics of 'Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measures'. Together they form a unique fingerprint.

    Cite this