Lecture 5

Prof. Henning Müller and Mara Graziani (PhD student)

University of Applied Sciences of Western Switzerland

Course Home

Evaluation of interpretability experiments

The physicist Richard Feynman stated “What I cannot create, I do not understand”. Shall we evaluate the interpretability of AI by testing whether a user could recreate the whole system? How to evaluate AI interpretability, in general? Multiple approaches were proposed in the literature, although large part of the development focused on evaluating interpretablity from the perspective of technical developemnt. In this class, we will discuss how to approach evaluation, presenting some examples on a medical application.

Class Outline

1. Motivation
2. Targeting the receivers of explanations
3. Ethical concerns and “empty explanations”
4. Adapting rather than applying: an example for medical image analysis
5. General metrics for evaluating explainable AI
6. Evaluation examples

Video Parte 1

Video Parte 2

Material

Hoffman, Robert R., et al. “Metrics for explainable AI: Challenges and prospects.” arXiv preprint arXiv:1812.04608 (2018).

Weller, Adrian. “Transparency: motivations and challenges.” Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Springer, Cham, 2019. 23-40.

Adebayo, Julius, et al. “Debugging Tests for Model Explanations.” arXiv preprint arXiv:2011.05429 (2020).

Assignments

A1.

(Optional) Evaluate your experiments!