Hint

To view additional information and to make edit suggestions, click the individual items.

Explainability

Objective

To ensure Model functions and outputs are explainable and justifiable as far as is practically reasonable in order to (a) foster explainability for Stakeholders, (b) promote Model trust, (c) facilitate Model debugging and understanding, and (d) promote compliance with existing laws and statutes.

16.1. Product Definition(s)

Objective

To (a) ensure the transparency of Product Definitions; (b) foster multi-stakeholder buy-in through explanations; and (c) reduce ethical risks in Product Definition(s) decision-making and Model Runs.

Item nr.	Item Name and Page	Control	Aim
16.1.1.	Explainability Aims	Having consideration for (a) Product Definition(s), (b) the explanations and/or transparency sought, (c) the Model adopted, and (d) datasets used, document and assess the explainability aims of the Model.	To (a) clearly document the explainability and transparency aims of the Model; and (b) highlight associated risks that might occur in the Product Lifecycle.
16.1.2.	Explainability Stakeholder	Document and assess the internal and external Stakeholders affected by the Model.	To identify the Model explainability Stakeholders; and (b) highlight associated risks that might occur in the Product Lifecycle.
16.1.3.	Explainability Risks Assessment	Document and assess the individual risks of failing to provide model explainability, inclusive of a legal liability and Explainability Stakeholders mistrust.	To identify the risks of failing to provide Model explainability; and (b) highlight associated risks that might occur in the Product Lifecycle.
16.1.4.	Legal Requirements for Interpretability	Document and assess any specific legal requirements for Explainability in consultation with legal experts..	To (a) ensure that minimum standards for explainability are met and legal risk is addressed; and (b) highlight associated risks that might occur in the Product Lifecycle.
16.1.5.	Explainability Requirements	Document and assess the explainability and transparency requirements and levels in light of (a) Explainability Aims, (b) Explainability Stakeholders, and (c) Explainability Risks, taking care that the elicitation of said requirements involves appropriate guidance, education and understanding of Stakeholders.	To (a) clearly document the explainability requirements of the Model; and (b) highlight associated risks that might occur in the Product Lifecycle.

16.2. Exploration

Objective

To identify and document Model explainability and transparency requirements, inclusive of Stakeholder needs.

Item nr.	Item Name and Page	Control	Aim
16.2.1.	Stakeholder Appraisal	Document and conduct (a) ad-hoc interviews, (b) structured surveys and/or (c) workshops with Explainability Stakeholders about their Model and Product concerns and literacy.	To (a) generate Explainability Stakeholders analytics in order to map Model explainability requirements and demands; and (b) highlight associated risks that might occur in the Product Lifecycle.
16.2.2.	Stakeholder Appraisal Analysis	Document, analyse and map the outcomes of the Stakeholder Appraisal against the Explainability Aims and Explainability Risks.	To (a) map and analyse Model explainability requirements and demands in light of the needs of Explainability Stakeholders; and (b) highlight associated risks that might occur in the Product Lifecycle.
16.2.3	Explainability Matrix	Document, assess, and derive a matrix evaluating and ranking the metrics and/or criteria of explanations needed for based on the (a) Stakeholder Appraisal Analyse, (b) Explainability Aims, (c) Explainability Risks, and (d) Explainability Requirements, inclusive of explanations accuracy, fidelity, consistency, stability, comprehensibility, certainty, and representativeness.	To (a) derive a clear matrix from which to assess Model explainability requirements; and (b) highlight associated risks that might occur in the Product Lifecycle.
16.2.4.	Explainability Feature Selection	Document and analyse the degree of Feature explainability needed in light of the Explainability Matrix.	To (a) identify the requisite degree of Feature explainability needed; and (b) highlight associated risks that might occur in the Product Lifecycle, such as later stage Model retraining due to feature ambiguity.
16.2.5.	Explainability Modelling Mapping & Analysis	Document and analyse the technical needs of Model explainability in light of the Explainability Matrix, inclusive of considerations of global vs. local explainability and/or pre-modelling explainability, modelling explainability, and post-hoc modelling explainability.	To (a) identify the technical needs of the Explainability Matrix; and (b) highlight associated risks that might occur in the Product Lifecycle.
16.2.6.	Explanation Frequency & Delivery Assessment	Document and assess the frequency, most suitable and practically reasonable methods of communicating Model explainability in light of the Explainability Matrix and Stakeholder Appraisal Analysis.	To (a) identify the most appropriate method of communicating Model explainability in order to promote explainability comprehension; and (b) highlight associated risks that might occur in the Product Lifecycle.

16.3. Development

Objective

To ensure that Model design represents the explainability requirements and demands of transparency aims as much as is reasonably practical.

Item nr.	Item Name and Page	Control	Aim
16.3.1.	Explainability Feature Selection Assessment	Conduct a Feature analysis of the Explainability Feature Selection in order to remove correlated and dependent Features.	To (a) interrogate the assumption of zero Feature dependency in explainability modeling; (b) prevent misleading Model explainability and transparency; and (c) highlight associated risks that might occur in the Product Lifecycle.
16.3.2.	Global Explainability Model Run	Document and run as many types of global explainability Models as is reasonably practical, such as Feature importances, Feature interactions, global surrogate Models, perturbation-based techniques or gradient-based techniques. When there is doubt about the stability of the techniques being used, test their quality through alternative parameterizations or by comparing across techniques.	To (a) generate global explainability of the model; (b) help promote model debugging; (c) ensure explainability fidelity and stability through numerous explainability model runs; and (d) highlight associated risks that might occur in the Product Lifecycle.
16.3.3.	Local Explainability Model Run	Document and run as many types of local explainability Models as is reasonably practical, such as perturbation-based techniques or gradient-based techniques or, for more specific examples, Local Interpretable Model-Agnostic Explanations (LIME), SHAP values, Anchor explanations amongst others. When there is doubt about the stability of the techniques being used, test their quality through alternative parameterizations or by comparing across techniques.	To (a) generate global explainability of the model; (b) help promote model debugging; (c) ensure explainability fidelity and stability through numerous explainability model runs; and (d) highlight associated risks that might occur in the Product Lifecycle.
16.3.4.	Visual Explanations Assessment	Develop visual aids to present and represent Model explainability and transparency insights, such as Tabular Graphics, Partial Dependency Plots, Individual Conditional Expectations, and/or Accumulated Local Effects plot.	To promote explainability comprehension.
16.3.5.	Example-based and Contrastive Explanations Assessment	Develop example-based and contrastive explanations to present and represent Model explainability insights, such as the underlying distribution of the data or select particular instances.	To promote explainability comprehension, such as of complex data distributions and/or datasets for Explainability audiences.

16.4. Production

Objective

To monitor and track the performance of the explanations and trigger when any of the explainability approaches need to be re-trained.

Item nr.	Item Name and Page	Control	Aim
16.4.1.	Explainability Model Thresholds	Set clear performance thresholds and limitations for explainability Model(s).	To (a) define parameters for the continued suitability and performance of explainability Model(s); and (b) highlight associated risks.
16.4.2.	Explainability Model Review & Monitoring	Periodically, or when significant Model changes occur, review implemented explainability Model(s) in light of Explainability Model Thresholds.	To (a) ensure the continued suitability and performance of explainability Model(s) and their explanations; and (b) highlight associated risks.
16.4.3.	Explanation Tracking & Monitoring	Document and conduct (a) ad-hoc interviews, (b) structured surveys, and/or (c) workshops with Explainability Stakeholders on explanations provided and adjust outcomes in Section 14 - Performance Robustness accordingly.	To ensure the continued effectiveness and suitability of provided Model explanations.