GruPaTo at SemEval-2020 Task 12: Retraining mBERT on Social Media and Fine-tuned Offensive Language Models

Tommaso Caselli; Davide Colla; Valerio Basile; Jelena Mitrović; Michael Granitzer

GruPaTo at SemEval-2020 Task 12: Retraining mBERT on Social Media and Fine-tuned Offensive Language Models

Tommaso Caselli
, Davide Colla
, Valerio Basile
, Jelena Mitrović
, Michael Granitzer

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

90 Downloads (Pure)

Abstract

We introduce an approach to multilingual Offensive Language Detection based on the mBERT transformer model. We download extra training data from Twitter in English, Danish, and Turkish, and use it to re-train the model. We then fine-tuned the model on the provided training data and, in some configurations, implement transfer learning approach exploiting the typological relatedness between English and Danish. Our systems obtained good results across the three languages (.9036 for EN, .7619 for DA, and .7789 for TR).

Original language	English
Title of host publication	Proceedings of the 14th International Workshop on Semantic Evaluation
Editors	Aurelie Herbelot, Xiaodan Zhu, Alexis Palmer, Nathan Schneider, Jonathan May, Ekaterina Shutova
Publisher	Association for Computational Linguistics (ACL)
Number of pages	9
Publication status	Published - 2020
Event	Fourteenth Workshop on Semantic Evaluation - Barcelona, Spain Duration: 12-Dec-2020 → 13-Dec-2020

Workshop

Workshop	Fourteenth Workshop on Semantic Evaluation
Abbreviated title	SemEval-2020
Country/Territory	Spain
City	Barcelona
Period	12/12/2020 → 13/12/2020

Keywords

offensive language
abusive language
BERT
CANCER-PATIENTS

Access to Document

offenseval_2020_camera_ready_fixedFinal publisher's version, 575 KBLicence: CC BY

Handle.net

Persistent link

1 Conference contribution

I Feel Offended, Don’t Be Abusive! Implicit/Explicit Messages in Offensive and Abusive Language
Caselli, T., Basile, V., Mitrović, J., Kartozya, I. & Granitzer, M., 2020, 12th International Conference on Language Resources and Evaluation. European Language Resources Association (ELRA), p. 6193-6202 12 p.
Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Open Access
File

Cite this

Caselli, T., Colla, D., Basile, V., Mitrović, J., & Granitzer, M. (2020). GruPaTo at SemEval-2020 Task 12: Retraining mBERT on Social Media and Fine-tuned Offensive Language Models. In A. Herbelot, X. Zhu, A. Palmer, N. Schneider, J. May, & E. Shutova (Eds.), Proceedings of the 14th International Workshop on Semantic Evaluation Association for Computational Linguistics (ACL).

Caselli, Tommaso ; Colla, Davide ; Basile, Valerio et al. / GruPaTo at SemEval-2020 Task 12 : Retraining mBERT on Social Media and Fine-tuned Offensive Language Models. Proceedings of the 14th International Workshop on Semantic Evaluation. editor / Aurelie Herbelot ; Xiaodan Zhu ; Alexis Palmer ; Nathan Schneider ; Jonathan May ; Ekaterina Shutova. Association for Computational Linguistics (ACL), 2020.

@inproceedings{7cd6338e7311459fb9c388f07ca58c3f,

title = "GruPaTo at SemEval-2020 Task 12: Retraining mBERT on Social Media and Fine-tuned Offensive Language Models",

abstract = "We introduce an approach to multilingual Offensive Language Detection based on the mBERT transformer model. We download extra training data from Twitter in English, Danish, and Turkish, and use it to re-train the model. We then fine-tuned the model on the provided training data and, in some configurations, implement transfer learning approach exploiting the typological relatedness between English and Danish. Our systems obtained good results across the three languages (.9036 for EN, .7619 for DA, and .7789 for TR).",

keywords = "offensive language, abusive language, BERT, CANCER-PATIENTS",

author = "Tommaso Caselli and Davide Colla and Valerio Basile and Jelena Mitrovi{\'c} and Michael Granitzer",

year = "2020",

language = "English",

editor = "Aurelie Herbelot and Xiaodan Zhu and Alexis Palmer and Nathan Schneider and Jonathan May and Ekaterina Shutova",

booktitle = "Proceedings of the 14th International Workshop on Semantic Evaluation",

publisher = "Association for Computational Linguistics (ACL)",

note = "Fourteenth Workshop on Semantic Evaluation, SemEval-2020 ; Conference date: 12-12-2020 Through 13-12-2020",

}

Caselli, T, Colla, D, Basile, V, Mitrović, J & Granitzer, M 2020, GruPaTo at SemEval-2020 Task 12: Retraining mBERT on Social Media and Fine-tuned Offensive Language Models. in A Herbelot, X Zhu, A Palmer, N Schneider, J May & E Shutova (eds), Proceedings of the 14th International Workshop on Semantic Evaluation. Association for Computational Linguistics (ACL), Fourteenth Workshop on Semantic Evaluation, Barcelona, Spain, 12/12/2020.

GruPaTo at SemEval-2020 Task 12: Retraining mBERT on Social Media and Fine-tuned Offensive Language Models. / Caselli, Tommaso; Colla, Davide; Basile, Valerio et al.
Proceedings of the 14th International Workshop on Semantic Evaluation. ed. / Aurelie Herbelot; Xiaodan Zhu; Alexis Palmer; Nathan Schneider; Jonathan May; Ekaterina Shutova. Association for Computational Linguistics (ACL), 2020.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - GruPaTo at SemEval-2020 Task 12

T2 - Fourteenth Workshop on Semantic Evaluation

AU - Caselli, Tommaso

AU - Colla, Davide

AU - Basile, Valerio

AU - Mitrović, Jelena

AU - Granitzer, Michael

PY - 2020

Y1 - 2020

N2 - We introduce an approach to multilingual Offensive Language Detection based on the mBERT transformer model. We download extra training data from Twitter in English, Danish, and Turkish, and use it to re-train the model. We then fine-tuned the model on the provided training data and, in some configurations, implement transfer learning approach exploiting the typological relatedness between English and Danish. Our systems obtained good results across the three languages (.9036 for EN, .7619 for DA, and .7789 for TR).

AB - We introduce an approach to multilingual Offensive Language Detection based on the mBERT transformer model. We download extra training data from Twitter in English, Danish, and Turkish, and use it to re-train the model. We then fine-tuned the model on the provided training data and, in some configurations, implement transfer learning approach exploiting the typological relatedness between English and Danish. Our systems obtained good results across the three languages (.9036 for EN, .7619 for DA, and .7789 for TR).

KW - offensive language

KW - abusive language

KW - BERT

KW - CANCER-PATIENTS

M3 - Conference contribution

BT - Proceedings of the 14th International Workshop on Semantic Evaluation

A2 - Herbelot, Aurelie

A2 - Zhu, Xiaodan

A2 - Palmer, Alexis

A2 - Schneider, Nathan

A2 - May, Jonathan

A2 - Shutova, Ekaterina

PB - Association for Computational Linguistics (ACL)

Y2 - 12 December 2020 through 13 December 2020

ER -

GruPaTo at SemEval-2020 Task 12: Retraining mBERT on Social Media and Fine-tuned Offensive Language Models

Abstract

Workshop

Keywords

Access to Document

Handle.net

Fingerprint

Research output

I Feel Offended, Don’t Be Abusive! Implicit/Explicit Messages in Offensive and Abusive Language

Cite this