Discovering the Rationale of Decisions: Experiments on Aligning Learning and Reasoning

Cor Steging, Silja Renooij, Bart Verheij

Research output: Working paperPreprintAcademic

21 Downloads (Pure)


In AI and law, systems that are designed for decision support should be explainable when pursuing justice. In order for these systems to be fair and responsible, they should make correct decisions and make them using a sound and transparent rationale. In this paper, we introduce a knowledge-driven method for model-agnostic rationale evaluation using dedicated test cases, similar to unit-testing in professional software development. We apply this new method in a set of machine learning experiments aimed at extracting known knowledge structures from artificial datasets from fictional and non-fictional legal settings. We show that our method allows us to analyze the rationale of black-box machine learning systems by assessing which rationale elements are learned or not. Furthermore, we show that the rationale can be adjusted using tailor-made training data based on the results of the rationale evaluation.
Original languageEnglish
Publication statusSubmitted - 14-May-2021

Publication series

PublisherCornell University Press
ISSN (Print)2331-8422


  • cs.AI
  • cs.LG

Cite this