PAT Workbench: Annotation and Evaluation of Text and Pictures in Multimodal Instructions

Ielka van der Sluis, Lennart Kloppenburg, Gisela Redeker

    OnderzoeksoutputAcademic

    80 Downloads (Pure)

    Samenvatting

    The PAT Workbench is an online tool to investigate the design of multimodal instructions (MIs), i.e., instructions that contain both pictures and text. The benefit of including pictures in information presentation has been established, but the characteristics of those pictures and of their textual counterparts and the relation(s) between them have not been researched in a systematic manner. We present the PAT Workbench, a tool to store, annotate and retrieve MIs based on a validated coding scheme with currently 42 categories that describe instructions in terms of textual features, pictorial elements, and picture-text relations. The PAT Workbench currently facilitates collaborative annotation and inter-annotator agreement calculation. Future work on the PAT Workbench includes expanding its functionality and usability by (i) making the MI annotation scheme dynamic for adding relevant features based on empirical evaluations of the MIs, (ii) implementing algorithms for automatic tagging of MI
    features, and (iii) implementing automatic MI evaluation algorithms based on results obtained via e.g. crowdsourced assessments of MIs.
    Originele taal-2English
    Aantal pagina's1
    StatusPublished - dec-2016
    EvenementLT for DH: Language Technology Resources and Tools for Digital Humanities - Osaka, Osaka, Japan
    Duur: 11-dec-201611-dec-2016
    https://www.clarin-d.net/en/current-issues/lt4dh

    Workshop

    WorkshopLT for DH: Language Technology Resources and Tools for Digital Humanities
    Verkorte titelLT4DH
    Land/RegioJapan
    StadOsaka
    Periode11/12/201611/12/2016
    Internet adres

    Citeer dit