Text-Picture Relations in Cooking Instructions

Ielka van der Sluis, Shadira Leito, Gisela Redeker

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review


    Like many other instructions, recipes on packages with ready-to-use ingredients for a dish combine a series of pictures with short text paragraphs. The information presentation in such multimodal instructions can be compact (either text or picture) and/or cohesive (text and picture). In an exploratory
    corpus study, 30 instructions for semi-prefabricated meals were annotated for text-picture relations. A slight majority of the 452 actions in the corpus were presented only textually. A third were presented in text and picture, indicating a moderate amount of cohesion. A minority of 31 actions (7%) were presented only pictorially, suggesting that the potential for compact multimodal
    presentation may be rather limited in these instructions.
    Original languageEnglish
    Title of host publicationProceedings of LREC 2016, Tenth International Conference on Language Resources and Evaluation
    Subtitle of host publicationProceedings of the Twelfth Joint ISO - ACL SIGSEM Workshop on Interoperable Semantic Annotation (ISA-12)
    EditorsHarry Bunt
    Place of PublicationPortoroz, Slovenia
    ISBN (Electronic)978-2-9517408-9-1
    Publication statusPublished - 28-May-2016


    • multimodal instruction
    • annotion
    • evaluation

    Cite this