Italian Transformers Under the Linguistic Lens

Alessio Miaschi*, Gabriele Sarti, Dominique Brunato, Felice Dell'Orletta, Giulia Venturi

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

In this paper we present an in-depth investigation of the linguistic knowledge encoded by the transformer models currently available for the Italian language. In particular, we investigate whether and how using different architectures of probing models affects the performance of Italian transformers in encoding a wide spectrum of linguistic features. Moreover, we explore how this implicit knowledge varies according to different textual genres.
Original languageEnglish
Title of host publicationProceedings of the Seventh Italian Conference on Computational Linguistics (CLiC-it 2020)
EditorsJohanna Monti, Felice Dell'Orletta, Fabio Tamburini
PublisherCEUR Workshop Proceedings (CEUR-WS.org)
Publication statusPublished - 1-Mar-2021
Externally publishedYes
EventItalian Conference on Computational Linguistics 2020 - Bologna, Italy
Duration: 1-Mar-20213-Mar-2021

Conference

ConferenceItalian Conference on Computational Linguistics 2020
Abbreviated titleCLiC-it 2020
Country/TerritoryItaly
CityBologna
Period01/03/202103/03/2021

Keywords

  • natural language processing
  • deep learning
  • probing task
  • interpretability
  • italian language
  • neural language models
  • transformers

Cite this