Rationale Discovery and Explainable AI

Cor Steging*, Silja Renooij, Bart Verheij

*Corresponding author voor dit werk

OnderzoeksoutputAcademicpeer review

7 Citaten (Scopus)
84 Downloads (Pure)


The justification of an algorithm's outcomes is important in many domains, and in particular in the law. However, previous research has shown that machine learning systems can make the right decisions for the wrong reasons: despite high accuracies, not all of the conditions that define the domain of the training data are learned. In this study, we investigate what the system does learn, using state-of-the-art explainable AI techniques. With the use of SHAP and LIME, we are able to show which features impact the decision making process and how the impact changes with different distributions of the training data. However, our results also show that even high accuracy and good relevant feature detection are no guarantee for a sound rationale. Hence these state-of-the-art explainable AI techniques cannot be used to fully expose unsound rationales, further advocating the need for a separate method for rationale evaluation.

Originele taal-2English
TitelLegal Knowledge and Information Systems - JURIX 2021
SubtitelThe 34th Annual Conference
RedacteurenErich Schweighofer
UitgeverijIOS Press
Aantal pagina's10
ISBN van elektronische versie9781643682525
StatusPublished - 2-dec.-2021
Evenement34th International Conference on Legal Knowledge and Information Systems, JURIX 2021 - Virtual, Online, Lithuania
Duur: 8-dec.-202110-dec.-2021

Publicatie series

NaamFrontiers in Artificial Intelligence and Applications
ISSN van geprinte versie0922-6389


Conference34th International Conference on Legal Knowledge and Information Systems, JURIX 2021
StadVirtual, Online


Duik in de onderzoeksthema's van 'Rationale Discovery and Explainable AI'. Samen vormen ze een unieke vingerafdruk.

Citeer dit