Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer

Huiyuan Lai; Antonio Toral Ruiz; Malvina Nissim

Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer

Huiyuan Lai, Antonio Toral Ruiz, Malvina Nissim

Computational Linguistics (CL)

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

13 Downloads (Pure)

Abstract

We exploit the pre-trained seq2seq model mBART for multilingual text style transfer. Using machine translated data as well as gold aligned English sentences yields state-of-the-art results in the three target languages we consider. Besides, in view of the general scarcity of parallel data, we propose a modular approach for multilingual formality transfer, which consists of two training strategies that target adaptation to both language and task. Our approach achieves competitive performance without monolingual task-specific parallel data and can be applied to other style transfer tasks as well as to other languages.

Original language	English
Title of host publication	Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics
Publisher	Association for Computational Linguistics, ACL Anthology
Publication status	E-pub ahead of print - 2022
Event	The 60th Annual Meeting of the Association for Computational Linguistics - Dublin, Ireland Duration: 22-May-2022 → 27-May-2022

Conference

Conference	The 60th Annual Meeting of the Association for Computational Linguistics
Country/Territory	Ireland
City	Dublin
Period	22/05/2022 → 27/05/2022

Access to Document

ACL2022_Multilingual_Text_Style_TransferFinal publisher's version, 711 KBLicence: Unspecified

Handle.net

Persistent link

Cite this

@inproceedings{d6da73a87b484729870f9d21d9059a34,

title = "Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer",

abstract = "We exploit the pre-trained seq2seq model mBART for multilingual text style transfer. Using machine translated data as well as gold aligned English sentences yields state-of-the-art results in the three target languages we consider. Besides, in view of the general scarcity of parallel data, we propose a modular approach for multilingual formality transfer, which consists of two training strategies that target adaptation to both language and task. Our approach achieves competitive performance without monolingual task-specific parallel data and can be applied to other style transfer tasks as well as to other languages.",

author = "Huiyuan Lai and {Toral Ruiz}, Antonio and Malvina Nissim",

year = "2022",

language = "English",

booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics",

publisher = "Association for Computational Linguistics, ACL Anthology",

note = "The 60th Annual Meeting of the Association for Computational Linguistics ; Conference date: 22-05-2022 Through 27-05-2022",

}

Lai, H , Toral Ruiz, A & Nissim, M 2022, Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer. in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, ACL Anthology, The 60th Annual Meeting of the Association for Computational Linguistics , Dublin, Ireland, 22/05/2022.

Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer. / Lai, Huiyuan ; Toral Ruiz, Antonio ; Nissim, Malvina.

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, ACL Anthology, 2022.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Multilingual Pre-training with Language and Task Adaptation for Multilingual Text Style Transfer

AU - Lai, Huiyuan

AU - Toral Ruiz, Antonio

AU - Nissim, Malvina

PY - 2022

Y1 - 2022

N2 - We exploit the pre-trained seq2seq model mBART for multilingual text style transfer. Using machine translated data as well as gold aligned English sentences yields state-of-the-art results in the three target languages we consider. Besides, in view of the general scarcity of parallel data, we propose a modular approach for multilingual formality transfer, which consists of two training strategies that target adaptation to both language and task. Our approach achieves competitive performance without monolingual task-specific parallel data and can be applied to other style transfer tasks as well as to other languages.

AB - We exploit the pre-trained seq2seq model mBART for multilingual text style transfer. Using machine translated data as well as gold aligned English sentences yields state-of-the-art results in the three target languages we consider. Besides, in view of the general scarcity of parallel data, we propose a modular approach for multilingual formality transfer, which consists of two training strategies that target adaptation to both language and task. Our approach achieves competitive performance without monolingual task-specific parallel data and can be applied to other style transfer tasks as well as to other languages.

M3 - Conference contribution

BT - Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics

PB - Association for Computational Linguistics, ACL Anthology

T2 - The 60th Annual Meeting of the Association for Computational Linguistics

Y2 - 22 May 2022 through 27 May 2022

ER -