Contrastive Language-Image Pre-training for the Italian Language

Federico Bianchi; Giuseppe Attanasio; Raphael Pisoni; Silvia Terragni; Gabriele Sarti; Sri Lakshimi

Contrastive Language-Image Pre-training for the Italian Language

Federico Bianchi^*, Giuseppe Attanasio, Raphael Pisoni, Silvia Terragni, Gabriele Sarti, Sri Lakshimi

^*Corresponding author for this work

Computational Linguistics (CL)

Research output: Working paper › Preprint › Academic

Abstract

CLIP (Contrastive Language-Image Pre-training) is a very recent multi-modal model that jointly learns representations of images and texts. The model is trained on a massive amount of English data and shows impressive performance on zero-shot classification tasks. Training the same model on a different language is not trivial, since data in other languages might be not enough and the model needs high-quality translations of the texts to guarantee a good performance. In this paper, we present the first CLIP model for the Italian Language (CLIP-Italian), trained on more than 1.4 million image-text pairs. Results show that CLIP-Italian outperforms the multilingual CLIP model on the tasks of image retrieval and zero-shot classification.

Original language	English
Publisher	arXiv
Publication status	Submitted - 19-Aug-2021

Publication series

Name	ArXiv
Publisher	Cornell University Press
ISSN (Print)	2331-8422

Keywords

contrastive learning
zero-shot image classification
deep learning
natural language processing
italian language
image retrieval

Access to Document

https://arxiv.org/pdf/2108.08688Licence: CC BY

Handle.net

Persistent link

The AI that explains images in Italian
Bianchi, F., Attanasio, G., Pisoni, R., Terragni, S., Sarti, G. & Lakshimi, S.
13/09/2021
1 item of Media coverage
Press/Media: Research › Academic
CLIP-Italian: A new AI model to connect images and texts in Italian
Bianchi, F., Attanasio, G., Pisoni, R., Terragni, S., Sarti, G. & Lakshimi, S.
26/08/2021
1 item of Media coverage
Press/Media: Research › Academic

Cite this

@techreport{92b9297fb32a4abf9b5214bf427c0519,

title = "Contrastive Language-Image Pre-training for the Italian Language",

abstract = "CLIP (Contrastive Language-Image Pre-training) is a very recent multi-modal model that jointly learns representations of images and texts. The model is trained on a massive amount of English data and shows impressive performance on zero-shot classification tasks. Training the same model on a different language is not trivial, since data in other languages might be not enough and the model needs high-quality translations of the texts to guarantee a good performance. In this paper, we present the first CLIP model for the Italian Language (CLIP-Italian), trained on more than 1.4 million image-text pairs. Results show that CLIP-Italian outperforms the multilingual CLIP model on the tasks of image retrieval and zero-shot classification.",

keywords = "contrastive learning, zero-shot image classification, deep learning, natural language processing, italian language, image retrieval",

author = "Federico Bianchi and Giuseppe Attanasio and Raphael Pisoni and Silvia Terragni and Gabriele Sarti and Sri Lakshimi",

year = "2021",

month = aug,

day = "19",

language = "English",

series = "ArXiv",

publisher = "arXiv",

type = "WorkingPaper",

institution = "arXiv",

}

TY - UNPB

T1 - Contrastive Language-Image Pre-training for the Italian Language

AU - Bianchi, Federico

AU - Attanasio, Giuseppe

AU - Pisoni, Raphael

AU - Terragni, Silvia

AU - Sarti, Gabriele

AU - Lakshimi, Sri

PY - 2021/8/19

Y1 - 2021/8/19

N2 - CLIP (Contrastive Language-Image Pre-training) is a very recent multi-modal model that jointly learns representations of images and texts. The model is trained on a massive amount of English data and shows impressive performance on zero-shot classification tasks. Training the same model on a different language is not trivial, since data in other languages might be not enough and the model needs high-quality translations of the texts to guarantee a good performance. In this paper, we present the first CLIP model for the Italian Language (CLIP-Italian), trained on more than 1.4 million image-text pairs. Results show that CLIP-Italian outperforms the multilingual CLIP model on the tasks of image retrieval and zero-shot classification.

AB - CLIP (Contrastive Language-Image Pre-training) is a very recent multi-modal model that jointly learns representations of images and texts. The model is trained on a massive amount of English data and shows impressive performance on zero-shot classification tasks. Training the same model on a different language is not trivial, since data in other languages might be not enough and the model needs high-quality translations of the texts to guarantee a good performance. In this paper, we present the first CLIP model for the Italian Language (CLIP-Italian), trained on more than 1.4 million image-text pairs. Results show that CLIP-Italian outperforms the multilingual CLIP model on the tasks of image retrieval and zero-shot classification.

KW - contrastive learning

KW - zero-shot image classification

KW - deep learning

KW - natural language processing

KW - italian language

KW - image retrieval

M3 - Preprint

T3 - ArXiv

BT - Contrastive Language-Image Pre-training for the Italian Language

PB - arXiv

ER -

Contrastive Language-Image Pre-training for the Italian Language

Abstract

Publication series

Keywords

Access to Document

Handle.net

Fingerprint

Press/Media

The AI that explains images in Italian

CLIP-Italian: A new AI model to connect images and texts in Italian

Cite this