Gamification for Word Sense Labeling

Noortje Venhuizen, Valerio Basile, Kilian Evang, Johan Bos

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    36 Citations (Scopus)
    81 Downloads (Pure)


    Obtaining gold standard data for word sense disambiguation is important but costly. We show how it can be done using a “Game with a Purpose” (GWAP) called Wordrobe. This game consists of a large set of multiple-choice questions on word senses generated from the Groningen Meaning Bank. The players need to answer these questions, scoring points depending on the agreement with fellow players. The working assumption is that the right sense for a word can be determined by the answers given by the players. To evaluate our method, we gold-standard tagged a portion of the data that was also used in the GWAP. A comparison yielded promising results, ranging from a precision of 0.88 and recall of 0.83 for relative majority agreement, to a precision of 0.98 and recall of 0.35 for questions that were answered unanimously.
    Original languageEnglish
    Title of host publicationProceedings of the 10th International Conference on Computational Semantics (IWCS'13) -- Short Papers
    EditorsKartin Erk, Alexander Koller
    Place of PublicationPotsdam, Germany
    Number of pages7
    Publication statusPublished - 2013

    Cite this