Hint

To view additional information and to make edit suggestions, click the individual items.

Model Decision-Making

Objective

To determine the most desirable and feasible model to achieve the desired Product Outcomes through consideration of several interacting analyses.

Item nr.	Item Name and Page	Control	Aim
5.1.	Model Type - Metric Fit Analysis	Document and assess the Model requirements needed to meet the Product Definitions, Outcome Definitions, and Product & Outcome Definitions Data & Model Metrics, as discussed in Section 4 - Problem Mapping.	To ensure that chosen Model(s) meet the requirements of the Product Definitions, Outcome Definitions, and Product & Outcome Definitions Data & Model Metrics.
5.2.	Model Type - Risk Analysis	Document and assess Model requirements needed to meet the Explainability Requirements and Product Risk Analysis, as discussed in Section 16 - Explainability; Section 4 - Problem Mapping.	To ensure that chosen Model(s) meet the requirements of the Explainability Requirements and Product Risk Analysis.
5.3.	Model Type - Organisation Analysis	Document and assess the compatibility of potential Models with the Organisation Capacity Analysis, Product Scaling Analysis, and Product Integration Strategy, as discussed in Section 4 - Problem Mapping, given technical considerations.	To ensure that chosen Model(s) meet the requirements of the Organisation Capacity Analysis, Product Scaling Analysis, and Product Integration Strategy.
5.4.	Model Type - Best Fit Analysis	Document and assess the most appropriate Models that best meet the requirements of, and which produces the most favorable outcome given the trade-offs between, the Model Type - Metric Fit, Risk and Organization Analyses.	To (a) ensure that the most appropriate Model(s) are chosen; and (b) highlight associated risks that might occur in the Product Lifecycle.
5.5.	Acceptance Criteria - Metrics	Document and define the desired performance for an acceptable Model in terms of clear Model and data metrics that are written from the end user's perspective.	To (a) determine the metrics and desired performance for an acceptable Model; and (b) highlight associated risks that might occur in the Product Lifecycle.
5.6.	Acceptance Criteria -Accuracy, Bias, and Fairness	Document and define clear, narrow accuracy goals and metrics that manage the tradeoff of accuracy and explainability. Document and define the Model requirements needed to meet the Fairness & Non-Discrimination goals, as discussed more thoroughly and technically in Section 11 - Fairness & Non-Discrimination.	To (a) ensure appropriate accuracy, bias and fairness metrics for Model(s); and (b) highlight associated risks that might occur in the Product Lifecycle.
5.7.	Acceptance Criteria - Error Rate Analysis	Consider the Societal and Industry Contexts in determining the acceptable method for error measurement, as discussed in Section 4 - Problem Mapping. Document and define the acceptable error types and rates for the Product as required by Representativeness & Specification, as discussed more thoroughly and technically in Section 13 - Representativeness & Specification. Analyze any potential tension between achievable and acceptable error rates and determine whether that tension can be resolved.	To (a) ensure appropriate error type and rate metrics for Model(s); and (b) highlight associated risks that might occur in the Product Lifecycle.
5.8.	Acceptance Criteria -Key Business Metrics / Targeted Metrics	Document and define the key business metrics (KPIs) as determined in Problem Statement & Solution Mapping, as discussed in Section 4 - Problem Mapping, and translate them into metrics that can be tracked within the framework of chosen Model(s), or into proxy metrics if direct tracking is not feasible.	To (a) ensure appropriate business metrics for Model(s); and (b) highlight associated risks that might occur in the Product Lifecycle.
5.9.	Technical Considerations	Document and assess technical issues that should be considered during the Model selection process.	To (a) ensure that technical issues are considered when selecting Models; and (b) highlight associated risks that might occur in the Product Lifecycle.