Pre-Trained Language-Meaning Models for Multilingual Parsing and Generation

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

80 Downloads (Pure)

Abstract

Pre-trained language models (PLMs) have achieved great success in NLP and have recently been used for tasks in computational semantics. However, these tasks do not fully benefit from PLMs since meaning representations are not explicitly included. We introduce multilingual pre-trained language-meaning models based on Discourse Representation Structures (DRSs), including meaning representations besides natural language texts in the same model, and design a new strategy to reduce the gap between the pre-training and fine-tuning objectives. Since DRSs are language neutral, cross-lingual transfer learning is adopted to further improve the performance of non-English tasks. Automatic evaluation results show that our approach achieves the best performance on both the multilingual DRS parsing and DRS-to-text generation tasks. Correlation analysis between automatic metrics and human judgements on the generation task further validates the effectiveness of our model. Human inspection reveals that out-of-vocabulary tokens are the main cause of erroneous results.
Original languageEnglish
Title of host publicationFindings of the Association for Computational Linguistics
Subtitle of host publicationACL 2023
EditorsAnna Rogers, Jordan Boyd-Graber, Naoaki Okazaki
PublisherAssociation for Computational Linguistics (ACL)
Pages5586–5600
Number of pages15
DOIs
Publication statusPublished - 2023

Fingerprint

Dive into the research topics of 'Pre-Trained Language-Meaning Models for Multilingual Parsing and Generation'. Together they form a unique fingerprint.

Cite this