Zero-shot Dependency Parsing with Pre-trained Multilingual Sentence Representations

    Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

    24 Downloads (Pure)


    We investigate whether off-the-shelf deep bidirectional sentence representations (Devlin et al., 2018) trained on a massively multilingual corpus (multilingual BERT) enable the development of an unsupervised universal dependency parser. This approach only leverages a mix of monolingual corpora in many languages and does not require any translation data making it applicable to low-resource languages. In our experiments we outperform the
    best CoNLL 2018 language-specific systems in all of the shared task’s six truly low-resource languages while using a single system. However, we also find that (i) parsing accuracy still varies dramatically when changing the training languages and (ii) in some target languages zero-shot transfer fails under all tested conditions, raising concerns on the ‘universality’ of the whole approach.
    Original languageEnglish
    Title of host publicationProceedings of the 2nd Workshop on Deep Learning Approaches for Low-Resource NLP (DeepLo 2019)
    Place of PublicationHong Kong, China
    PublisherAssociation for Computational Linguistics (ACL)
    Number of pages8
    Publication statusPublished - Nov-2019

    Cite this