08-13, 12:00–12:40 (Asia/Yerevan), 113W PAB
Explainable Artificial Intelligence (XAI) is crucial for the development of responsible, trustworthy AI. Machine learning models such as deep neural networks can perform highly complex computational tasks at scale, but they do not reveal their decision-making process. This becomes problematic when such models are used to make high-stakes decisions, such as medical diagnoses, which require clear explanations in order to be trusted.
This talk discusses Explainable AI using examples of interest for both machine learning practitioners and non-technical audiences. This talk is not very technical; it does not focus on how to apply an existing method to their model. Rather, the talk discusses the problem of Explainability_ as whole, namely: what is the Explainability Problem and why it must be solved, how recent academic literature addresses the problem, and how the problem will evolve with new legislation.
To get the most from this talk, the audience should have some familiarity with standard machine learning algorithms. However, no technical background is needed to grasp the key takeaways: the necessity of explainability in machine learning, the challenges of developing explainability methods, and the impact that XAI has on businesses, practitioners and end-users.
When a machine learning model does not meet the objectives of its deployment settings, there arises a demand for interpretability [1]. Practitioners and stakeholders would thus desire more information from a model than what can be gleaned from evaluation metrics; they need to know why a model makes its decisions, especially when its output is not in line with human intuition.
The increasing use of black-box models, which provide no transparency on their decision-making process, for high-stakes tasks has lead to the development of Explainable AI (XAI): a set of methods that help humans understand the outputs of machine learning models. [2] The number of papers written about XAI have increased dramatically in the past few years, and new open-source implementations of popular methods are continuously introduced. Yet, the Explainability problem is far from being solved.
This talk discusses Explainable AI using examples of interest for both machine learning practitioners and non-technical stakeholders, including business leaders and students of AI ethics. The focus of this talk is not on one particular method, nor does it provide code examples. Rather, it focuses on the problem of Explainability: that, even with a plethora of new XAI papers, and off-the-shelf explainers, a practitioner should think critically about their particular use case before selecting, applying and interpreting an explainer.
The talk is divided into three parts:
(1) The Explainability Problem (minutes 0-10).
- Definitions of key concepts: explainability, the transparency-utility tradeoff
- Real-world examples for why explainability is needed
- Survey of the XAI landscape
Key takeaway: when deploying machine learning models in high-stakes applications, human-friendly explanations are imperative.
(2) The Problem with Explainability (minutes 10-20).
We discuss three large problems with XAI:
- The gap in interpretation of explainers
- The unmet need of standardized, quality metrics for explanations
- The disagreement problem between different explainers
Key Takeaway: while there exist a large number of explanation methods, there remain some crucial open problems in the XAI landscape.
(3) The future of explainable AI: open questions, research, and regulation (minutes 20-30).
We focus on the upcoming EU AI Act [3], which will eventually define standards of explainability in the EU.
- What requirements could exist for a model to be deemed explainable ?
- What actions machine learning practitioners, leaders of data-centric businesses, and students should take towards the responsible development of AI
Key Takeaway: Legislative developments on safe AI standards consider explainability a essential for all AI systems.
No previous knowledge expected
Nura Kawa is a Research Scientist at neurocat (Berlin, Germany), a startup that delivers innovation in the development of safe and secure AI systems. Her current research focus is on adversarial robustness of deep neural networks. Additionally, she is interested in privacy-preserving machine learning and in explainable AI. Nura holds an MSc in Statistics and Data Science from KU Leuven (Leuven, Belgium) and a BA in Statistics from UC Berkeley (Berkeley, USA).