UmBERTo-MTSA@ AcCompl-It: Improving Complexity and Acceptability Prediction with Multi-task Learning on Self-Supervised Annotations

OnderzoeksoutputAcademicpeer review

Samenvatting

This work describes a self-supervised data augmentation approach used to improve learning models' performances when only a moderate amount of labeled data is available. Multiple copies of the original model are initially trained on the downstream task. Their predictions are then used to annotate a large set of unlabeled examples. Finally, multi-task training is performed on the parallel annotations of the resulting training set, and final scores are obtained by averaging annotator-specific head predictions. Neural language models are fine-tuned using this procedure in the context of the AcCompl-it shared task at EVALITA 2020, obtaining considerable improvements in prediction quality.
Originele taal-2English
TitelProceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2020)
RedacteurenValerio Basile, Danilo Croce, Maria Di Maro, Lucia Passaro
Plaats van productieOnline
UitgeverijCEUR Workshop Proceedings (CEUR-WS.org)
StatusPublished - 17-dec-2020
Extern gepubliceerdJa
EvenementEvaluation Campaign of Natural Language Processing and Speech Tools for Italian - Online
Duur: 17-dec-2020 → …
Congresnummer: 7

Workshop

WorkshopEvaluation Campaign of Natural Language Processing and Speech Tools for Italian
Verkorte titelEVALITA 2020
Periode17/12/2020 → …

Citeer dit