Evaluating Algorithms for the Generation of Referring Expressions: Going beyond Toy Domains.

Ielka van der Sluis, A. Gatt, K. van Deemter

    OnderzoeksoutputAcademicpeer review

    13 Citaten (Scopus)
    84 Downloads (Pure)

    Samenvatting

    We describe a corpus-based evaluation method-
    ology, applied to a number of classic algorithms
    in the generation of referring expressions. Fol-
    lowing up on earlier work involving very simple
    domains, this paper deals with the issues asso-
    ciated with domains that contain ‘real-life’ ob-
    jects of some complexity. Results indicate that
    state of the art algorithms perform very differ-
    ently when applied to a complex domain. More-
    over, if a version of the Incremental Algorithm is
    used then it becomes of huge importance to se-
    lect a good preference order, because some pref-
    erence orders are prone to generating very un-
    natural output. Finding good preference orders,
    however, can be difficult, as we show. These re-
    sults should contribute to a growing debate on
    the evaluation of nlg systems, arguing in favour
    of carefully constructed balanced and semanti-
    cally transparent corpora.
    Originele taal-2English
    TitelProceedings of the RANLP-2007
    Plaats van productieBorovets, Bulgaria
    StatusPublished - 2007

    Citeer dit