Synonym-Based Essay Generation and Augmentation for Robust Automatic Essay Scoring

Tsegaye Misikir Tashu*, Tomáš Horváth

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

Automatic essay scoring (AES) models based on neural networks (NN) have had a lot of success. However, research has shown that NN-based AES models have robustness issues, such that the output of a model changes easily with small changes in the input. We proposed to use keyword-based lexical substitution using BERT that generates new essays (adversarial samples) which are lexically similar to the original essay to evaluate the robustness of AES models trained on the original set. In order to evaluate the proposed approach, we implemented three NN-based scoring approaches and trained the scoring models using two stages. First, we trained each model using the original data and evaluate the performance using the original test and newly generated test set to see the impact of the adversarial sample of the model. Secondly, we trained the models by augmenting the generated adversarial essay with the original data to train a robust model against synonym-based adversarial attacks. The results of our experiments showed that extracting the most important words from the essay and replacing them with lexically similar words, as well as generating adversarial samples for augmentation, can significantly improve the generalization of NN-based AES models. Our experiments also demonstrated that the proposed defense is capable of not only defending against adversarial attacks, but also of improving the performance of NN-based AES models.

Original languageEnglish
Title of host publicationIntelligent Data Engineering and Automated Learning – IDEAL 2022 - 23rd International Conference, IDEAL 2022, Proceedings
EditorsHujun Yin, David Camacho, Peter Tino
PublisherSpringer Science and Business Media Deutschland GmbH
Pages12-21
Number of pages10
ISBN (Print)9783031217524
DOIs
Publication statusPublished - 2022
Externally publishedYes
Event23rd International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2022 - Manchester, United Kingdom
Duration: 24-Nov-202226-Nov-2022

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume13756 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference23rd International Conference on Intelligent Data Engineering and Automated Learning, IDEAL 2022
Country/TerritoryUnited Kingdom
CityManchester
Period24/11/202226/11/2022

Keywords

  • Adversarial attack
  • Automatic essay scoring
  • Data augmentation

Fingerprint

Dive into the research topics of 'Synonym-Based Essay Generation and Augmentation for Robust Automatic Essay Scoring'. Together they form a unique fingerprint.

Cite this