Crowdsourcing StoryLines: Harnessing the Crowd for Causal Relation Annotation

Tomasso Caselli, Oana Inel

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    183 Downloads (Pure)

    Abstract

    This paper describes a crowdsourcing experiment on the annotation of plot-like structures in En- glish news articles. The CrowdTruth methodology and metrics have been applied to select valid annotations from the crowd. We further run an in-depth analysis of the annotated data by compar- ing it with available expert data. Our results show a valuable use of crowdsourcing annotations for such complex semantic tasks, and promote a new annotation approach that combines crowd and experts.
    Original languageEnglish
    Title of host publicationEvents and Stories in the News
    Subtitle of host publicationProceedings of the Workshop
    EditorsTomasso Caselli, Ben Miller, Marieke van Erp, Piek Vossen, Martha Palmer, Eduard Hovy, Teruko Mitamura, David Caswell, Susan W. Brown, Claire Bonial
    PublisherAssociation for Computational Linguistics (ACL)
    Pages44-54
    Number of pages10
    Publication statusPublished - 2018
    EventEvents and Stories in the News - COLING 2018, Santa Fe, United States
    Duration: 20-Aug-201820-Aug-2018
    http://eventstory.news

    Workshop

    WorkshopEvents and Stories in the News
    Abbreviated titleEventStory 2018
    Country/TerritoryUnited States
    CitySanta Fe
    Period20/08/201820/08/2018
    Internet address

    Keywords

    • storyline
    • event extraction
    • crowdsourcing

    Fingerprint

    Dive into the research topics of 'Crowdsourcing StoryLines: Harnessing the Crowd for Causal Relation Annotation'. Together they form a unique fingerprint.

    Cite this