Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer

Huiyuan Lai; Antonio Toral Ruiz; Malvina Nissim

doi:10.18653/v1/2021.acl-short.62

Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer

Huiyuan Lai, Antonio Toral Ruiz, Malvina Nissim

Onderzoeksoutput › Academic › peer review

30 Citaten (Scopus)

131 Downloads (Pure)

Samenvatting

Scarcity of parallel data causes formality style transfer models to have scarce success in preserving content. We show that fine-tuning pre-trained language (GPT-2) and sequence-to-sequence (BART) models boosts content preservation, and that this is possible even with limited amounts of parallel data. Augmenting these models with rewards that target style and content –the two core aspects of the task– we achieve a new state-of-the-art.

Originele taal-2	English
Titel	Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing
Redacteuren	Chengqing Zong, Fei Xia, Wenjie Li, Roberto Navigli
Plaats van productie	Bangkok, Thailand
Uitgeverij	Association for Computational Linguistics, ACL Anthology
Pagina's	484-494
Aantal pagina's	11
Volume	2
DOI's	https://doi.org/10.18653/v1/2021.acl-short.62
Status	Published - 2021

Toegang tot document

10.18653/v1/2021.acl-short.62Licentie: CC BY

Thank you BART! Rewarding Pre-Trained Models Improves Formality Style TransferFinal publisher's version, 453 KBLicentie: CC BY

Handle.net

Permanente koppeling

Citeer dit

Lai, H., Toral Ruiz, A., & Nissim, M. (2021). Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer. In C. Zong, F. Xia, W. Li, & R. Navigli (editors), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Vol. 2, blz. 484-494). Association for Computational Linguistics, ACL Anthology. https://doi.org/10.18653/v1/2021.acl-short.62

Lai, Huiyuan ; Toral Ruiz, Antonio ; Nissim, Malvina. / Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. redacteur / Chengqing Zong ; Fei Xia ; Wenjie Li ; Roberto Navigli. Vol. 2 Bangkok, Thailand : Association for Computational Linguistics, ACL Anthology, 2021. blz. 484-494

@inproceedings{81df322b1fb6475d9036a75460301a7c,

title = "Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer",

abstract = "Scarcity of parallel data causes formality style transfer models to have scarce success in preserving content. We show that fine-tuning pre-trained language (GPT-2) and sequence-to-sequence (BART) models boosts content preservation, and that this is possible even with limited amounts of parallel data. Augmenting these models with rewards that target style and content –the two core aspects of the task– we achieve a new state-of-the-art.",

author = "Huiyuan Lai and {Toral Ruiz}, Antonio and Malvina Nissim",

year = "2021",

doi = "10.18653/v1/2021.acl-short.62",

language = "English",

volume = "2",

pages = "484--494",

editor = "Chengqing Zong and Fei Xia and Wenjie Li and Roberto Navigli",

booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing",

publisher = "Association for Computational Linguistics, ACL Anthology",

}

Lai, H , Toral Ruiz, A & Nissim, M 2021, Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer. in C Zong, F Xia, W Li & R Navigli (uitgave), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. vol. 2, Association for Computational Linguistics, ACL Anthology, Bangkok, Thailand, blz. 484-494. https://doi.org/10.18653/v1/2021.acl-short.62

Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer. / Lai, Huiyuan ; Toral Ruiz, Antonio ; Nissim, Malvina.
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. uitgave / Chengqing Zong; Fei Xia; Wenjie Li; Roberto Navigli. Vol. 2 Bangkok, Thailand: Association for Computational Linguistics, ACL Anthology, 2021. blz. 484-494.

Onderzoeksoutput › Academic › peer review

TY - GEN

T1 - Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer

AU - Lai, Huiyuan

AU - Toral Ruiz, Antonio

AU - Nissim, Malvina

PY - 2021

Y1 - 2021

N2 - Scarcity of parallel data causes formality style transfer models to have scarce success in preserving content. We show that fine-tuning pre-trained language (GPT-2) and sequence-to-sequence (BART) models boosts content preservation, and that this is possible even with limited amounts of parallel data. Augmenting these models with rewards that target style and content –the two core aspects of the task– we achieve a new state-of-the-art.

AB - Scarcity of parallel data causes formality style transfer models to have scarce success in preserving content. We show that fine-tuning pre-trained language (GPT-2) and sequence-to-sequence (BART) models boosts content preservation, and that this is possible even with limited amounts of parallel data. Augmenting these models with rewards that target style and content –the two core aspects of the task– we achieve a new state-of-the-art.

U2 - 10.18653/v1/2021.acl-short.62

DO - 10.18653/v1/2021.acl-short.62

M3 - Conference contribution

VL - 2

SP - 484

EP - 494

BT - Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing

A2 - Zong, Chengqing

A2 - Xia, Fei

A2 - Li, Wenjie

A2 - Navigli, Roberto

PB - Association for Computational Linguistics, ACL Anthology

CY - Bangkok, Thailand

ER -

Lai H , Toral Ruiz A , Nissim M. Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer. In Zong C, Xia F, Li W, Navigli R, redacteurs, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Vol. 2. Bangkok, Thailand: Association for Computational Linguistics, ACL Anthology. 2021. blz. 484-494 doi: 10.18653/v1/2021.acl-short.62

Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer

Samenvatting

Toegang tot document

Handle.net

Vingerafdruk

Citeer dit