MIND at SemEval-2023 Task 11: From Uncertain Predictions to Subjective Disagreement

Giulia Rizzi; Alessandro Astorino; Daniel Scalena; Paolo Rosso; Elisabetta Fersini

doi:10.18653/v1/2023.semeval-1.77

MIND at SemEval-2023 Task 11: From Uncertain Predictions to Subjective Disagreement

Giulia Rizzi
, Alessandro Astorino
, Daniel Scalena
, Paolo Rosso
, Elisabetta Fersini

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

3 Citations (Scopus)

Abstract

This paper describes the participation of the research laboratory MIND, at the University of Milano-Bicocca, in the SemEval 2023 task related to Learning With Disagreements (Le-Wi-Di). The main goal is to identify the level of agreement/disagreement from a collection of textual datasets with different characteristics in terms of style, language, and task. The proposed approach is grounded on the hypothesis that the disagreement between annotators could be grasped by the uncertainty that a model, based on several linguistic characteristics, could have on the prediction of a given gold label.

Original language	English
Title of host publication	Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)
Editors	Atul Kr. Ojha, A. Seza Dogruoz, Giovanni Da San Martino, Harish Tayyar Madabushi, Ritesh Kumar, Elisa Sartori
Publisher	Association for Computational Linguistics, ACL Anthology
Pages	556-564
Number of pages	9
ISBN (Electronic)	9781959429999
DOIs	https://doi.org/10.18653/v1/2023.semeval-1.77
Publication status	Published - 2023
Externally published	Yes
Event	17th International Workshop on Semantic Evaluation, SemEval 2023, co-located with the 61st Annual Meeting of the Association for Computational Linguistics, ACL 2023 - Hybrid, Toronto, Canada Duration: 13-Jul-2023 → 14-Jul-2023

Conference

Conference	17th International Workshop on Semantic Evaluation, SemEval 2023, co-located with the 61st Annual Meeting of the Association for Computational Linguistics, ACL 2023
Country/Territory	Canada
City	Hybrid, Toronto
Period	13/07/2023 → 14/07/2023

Access to Document

10.18653/v1/2023.semeval-1.77Licence: CC BY

Handle.net

Persistent link

Cite this

Rizzi, G., Astorino, A., Scalena, D., Rosso, P., & Fersini, E. (2023). MIND at SemEval-2023 Task 11: From Uncertain Predictions to Subjective Disagreement. In A. K. Ojha, A. S. Dogruoz, G. Da San Martino, H. T. Madabushi, R. Kumar, & E. Sartori (Eds.), Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023) (pp. 556-564). Association for Computational Linguistics, ACL Anthology. https://doi.org/10.18653/v1/2023.semeval-1.77

Rizzi, Giulia ; Astorino, Alessandro ; Scalena, Daniel et al. / MIND at SemEval-2023 Task 11 : From Uncertain Predictions to Subjective Disagreement. Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023). editor / Atul Kr. Ojha ; A. Seza Dogruoz ; Giovanni Da San Martino ; Harish Tayyar Madabushi ; Ritesh Kumar ; Elisa Sartori. Association for Computational Linguistics, ACL Anthology, 2023. pp. 556-564

@inproceedings{1e60f4493131472e9b78eff8acb96391,

title = "MIND at SemEval-2023 Task 11: From Uncertain Predictions to Subjective Disagreement",

abstract = "This paper describes the participation of the research laboratory MIND, at the University of Milano-Bicocca, in the SemEval 2023 task related to Learning With Disagreements (Le-Wi-Di). The main goal is to identify the level of agreement/disagreement from a collection of textual datasets with different characteristics in terms of style, language, and task. The proposed approach is grounded on the hypothesis that the disagreement between annotators could be grasped by the uncertainty that a model, based on several linguistic characteristics, could have on the prediction of a given gold label.",

author = "Giulia Rizzi and Alessandro Astorino and Daniel Scalena and Paolo Rosso and Elisabetta Fersini",

note = "Publisher Copyright: {\textcopyright} 2023 Association for Computational Linguistics.; 17th International Workshop on Semantic Evaluation, SemEval 2023, co-located with the 61st Annual Meeting of the Association for Computational Linguistics, ACL 2023 ; Conference date: 13-07-2023 Through 14-07-2023",

year = "2023",

doi = "10.18653/v1/2023.semeval-1.77",

language = "English",

pages = "556--564",

editor = "Ojha, \{Atul Kr.\} and Dogruoz, \{A. Seza\} and \{Da San Martino\}, Giovanni and Madabushi, \{Harish Tayyar\} and Ritesh Kumar and Elisa Sartori",

booktitle = "Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)",

publisher = "Association for Computational Linguistics, ACL Anthology",

}

Rizzi, G, Astorino, A, Scalena, D, Rosso, P & Fersini, E 2023, MIND at SemEval-2023 Task 11: From Uncertain Predictions to Subjective Disagreement. in AK Ojha, AS Dogruoz, G Da San Martino, HT Madabushi, R Kumar & E Sartori (eds), Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023). Association for Computational Linguistics, ACL Anthology, pp. 556-564, 17th International Workshop on Semantic Evaluation, SemEval 2023, co-located with the 61st Annual Meeting of the Association for Computational Linguistics, ACL 2023, Hybrid, Toronto, Canada, 13/07/2023. https://doi.org/10.18653/v1/2023.semeval-1.77

MIND at SemEval-2023 Task 11: From Uncertain Predictions to Subjective Disagreement. / Rizzi, Giulia; Astorino, Alessandro; Scalena, Daniel et al.
Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023). ed. / Atul Kr. Ojha; A. Seza Dogruoz; Giovanni Da San Martino; Harish Tayyar Madabushi; Ritesh Kumar; Elisa Sartori. Association for Computational Linguistics, ACL Anthology, 2023. p. 556-564.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - MIND at SemEval-2023 Task 11

T2 - 17th International Workshop on Semantic Evaluation, SemEval 2023, co-located with the 61st Annual Meeting of the Association for Computational Linguistics, ACL 2023

AU - Rizzi, Giulia

AU - Astorino, Alessandro

AU - Scalena, Daniel

AU - Rosso, Paolo

AU - Fersini, Elisabetta

PY - 2023

Y1 - 2023

N2 - This paper describes the participation of the research laboratory MIND, at the University of Milano-Bicocca, in the SemEval 2023 task related to Learning With Disagreements (Le-Wi-Di). The main goal is to identify the level of agreement/disagreement from a collection of textual datasets with different characteristics in terms of style, language, and task. The proposed approach is grounded on the hypothesis that the disagreement between annotators could be grasped by the uncertainty that a model, based on several linguistic characteristics, could have on the prediction of a given gold label.

AB - This paper describes the participation of the research laboratory MIND, at the University of Milano-Bicocca, in the SemEval 2023 task related to Learning With Disagreements (Le-Wi-Di). The main goal is to identify the level of agreement/disagreement from a collection of textual datasets with different characteristics in terms of style, language, and task. The proposed approach is grounded on the hypothesis that the disagreement between annotators could be grasped by the uncertainty that a model, based on several linguistic characteristics, could have on the prediction of a given gold label.

UR - https://www.scopus.com/pages/publications/85160928691

U2 - 10.18653/v1/2023.semeval-1.77

DO - 10.18653/v1/2023.semeval-1.77

M3 - Conference contribution

AN - SCOPUS:85160928691

SP - 556

EP - 564

BT - Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)

A2 - Ojha, Atul Kr.

A2 - Dogruoz, A. Seza

A2 - Da San Martino, Giovanni

A2 - Madabushi, Harish Tayyar

A2 - Kumar, Ritesh

A2 - Sartori, Elisa

PB - Association for Computational Linguistics, ACL Anthology

Y2 - 13 July 2023 through 14 July 2023

ER -

Rizzi G, Astorino A, Scalena D, Rosso P, Fersini E. MIND at SemEval-2023 Task 11: From Uncertain Predictions to Subjective Disagreement. In Ojha AK, Dogruoz AS, Da San Martino G, Madabushi HT, Kumar R, Sartori E, editors, Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023). Association for Computational Linguistics, ACL Anthology. 2023. p. 556-564 doi: 10.18653/v1/2023.semeval-1.77

MIND at SemEval-2023 Task 11: From Uncertain Predictions to Subjective Disagreement

Abstract

Conference

Access to Document

Handle.net

Other files and links

Fingerprint

Cite this