Probing Linguistic Knowledge in Italian Neural Language Models across Language Varieties

Alessio Miaschi; Gabriele Sarti; Dominique Brunato; Felice Dell’Orletta; Giulia Venturi

doi:10.4000/ijcol.965

Probing Linguistic Knowledge in Italian Neural Language Models across Language Varieties

Alessio Miaschi^*, Gabriele Sarti, Dominique Brunato, Felice Dell’Orletta, Giulia Venturi

^*Bijbehorende auteur voor dit werk

Computational Linguistics (CL)

Onderzoeksoutput › Academic › peer review

11 Downloads (Pure)

Samenvatting

In this paper, we present an in-depth investigation of the linguistic knowledge encoded by the transformer models currently available for the Italian language. In particular, we investigate how the complexity of two different architectures of probing models affects the performance of the Transformers in encoding a wide spectrum of linguistic features. Moreover, we explore how this implicit knowledge varies according to different textual genres and language varieties.

Originele taal-2	English
Artikelnummer	2
Pagina's (van-tot)	25-44
Aantal pagina's	20
Tijdschrift	Italilan Journal of Computational Linguistics
Volume	8
Nummer van het tijdschrift	1
DOI's	https://doi.org/10.4000/ijcol.965
Status	Published - jul.-2022

Toegang tot document

10.4000/ijcol.965Licentie: CC BY-NC-ND

ijcol-965Final publisher's version, 1,04 MBLicentie: CC BY-NC-ND

Handle.net

Permanente koppeling

1 Conference contribution

Italian Transformers Under the Linguistic Lens
Miaschi, A., Sarti, G., Brunato, D., Dell'Orletta, F. & Venturi, G., 1-mrt.-2021, Proceedings of the Seventh Italian Conference on Computational Linguistics (CLiC-it 2020). Monti, J., Dell'Orletta, F. & Tamburini, F. (redactie). CEUR Workshop Proceedings (CEUR-WS.org)
Onderzoeksoutput › Academic › peer review

Open Access

Citeer dit

@article{d90a44d1f77f46bdac7646383e971116,

title = "Probing Linguistic Knowledge in Italian Neural Language Models across Language Varieties",

abstract = "In this paper, we present an in-depth investigation of the linguistic knowledge encoded by the transformer models currently available for the Italian language. In particular, we investigate how the complexity of two different architectures of probing models affects the performance of the Transformers in encoding a wide spectrum of linguistic features. Moreover, we explore how this implicit knowledge varies according to different textual genres and language varieties.",

keywords = "italian language, natural language processing, syntax, neural language models, language modeling, interpretability, interpretable machine learning",

author = "Alessio Miaschi and Gabriele Sarti and Dominique Brunato and Felice Dell{\textquoteright}Orletta and Giulia Venturi",

year = "2022",

month = jul,

doi = "10.4000/ijcol.965",

language = "English",

volume = "8",

pages = "25--44",

journal = "Italilan Journal of Computational Linguistics",

issn = "2499-4553",

number = "1",

}

TY - JOUR

T1 - Probing Linguistic Knowledge in Italian Neural Language Models across Language Varieties

AU - Miaschi, Alessio

AU - Sarti, Gabriele

AU - Brunato, Dominique

AU - Dell’Orletta, Felice

AU - Venturi, Giulia

PY - 2022/7

Y1 - 2022/7

N2 - In this paper, we present an in-depth investigation of the linguistic knowledge encoded by the transformer models currently available for the Italian language. In particular, we investigate how the complexity of two different architectures of probing models affects the performance of the Transformers in encoding a wide spectrum of linguistic features. Moreover, we explore how this implicit knowledge varies according to different textual genres and language varieties.

AB - In this paper, we present an in-depth investigation of the linguistic knowledge encoded by the transformer models currently available for the Italian language. In particular, we investigate how the complexity of two different architectures of probing models affects the performance of the Transformers in encoding a wide spectrum of linguistic features. Moreover, we explore how this implicit knowledge varies according to different textual genres and language varieties.

KW - italian language

KW - natural language processing

KW - syntax

KW - neural language models

KW - language modeling

KW - interpretability

KW - interpretable machine learning

U2 - 10.4000/ijcol.965

DO - 10.4000/ijcol.965

M3 - Article

VL - 8

SP - 25

EP - 44

JO - Italilan Journal of Computational Linguistics

JF - Italilan Journal of Computational Linguistics

SN - 2499-4553

IS - 1

M1 - 2

ER -

Probing Linguistic Knowledge in Italian Neural Language Models across Language Varieties

Samenvatting

Toegang tot document

Handle.net

Onderzoekersoutput

Italian Transformers Under the Linguistic Lens

Citeer dit