Hint

To view additional information and to make edit suggestions, click the individual items.

Project Checkpoints

Objective

To ensure that necessary factors are considered at key decision points.

Item nr.	Item Name and Page	Control	Aim
10.1.	Machine Learning Appropriate Tool Analysis	The Product Team should work cross-functionally with Stakeholders to define and document a Machine Learning checklist that considers the following areas, amongst other things: (a) Is there a different approach that will generate a greater return more quickly; (b) Given the results of the Data Capacity Analysis, does the Organisation have enough secure, non-discriminatory, representative, high quality data for every stage of the process; (c) Can the problem be solved by simple rules; (d) Does the Product solution require emotional intelligence or empathy; (e) Does the Product solution need to be fully interpretable or explainable; (f) Given the results of the Organisation Capacity Analysis, does the Organization have the people, processes, and tools necessary to productize the end product; (g) Can the consequences of Product failure be easily fixed or mitigated; and/or (h) What other non-technical solutions can be used to augment the Product and its offering and/or, more directly, whether Machine Learning is the best solution for the Product at hand.	To (a) ensure that Machine Learning is the appropriate method for solving the chosen problem; and (b) highlight associated risks that might occur in the Product Lifecycle.
10.2.	Data Buy v. Build Analysis	The Product Team should work cross-functionally with relevant Stakeholders to define and document a Buy v. Build checklist that considers the following areas: (a) Does the Organisation have enough data for every stage of the process (training, POC, production) and for every purpose (replacing stale/flawed data, measuring success); (b) Does the Organisation have the right type of data for every stage of the process (training, POC, production) and for every purpose (replacing stale/flawed data, measuring success); (c) Is bought data secure and free of privacy concerns; (d) Is the bias in the bought data limited, mitigatable, or removable; (e) Given the results of the Data Quality Analysis, does the Organisation have quality data and are datasets complete; (f) Given the Product Team Composition, does the Organisation have the staffing and expertise to clean, prepare, and maintain internal data; and/or (g) Given the Data Capacity Analysis, is the necessary data easily and readily available internally.	To (a) ensure that the Organisation's decision to either purchase data or utilize in-house data is appropriate based on Organisation capacity and/or constraints; and (b) highlight associated risks that might occur in the Product Lifecycle.
10.3.	Model Buy v. Build Analysis	The Product Team should work cross-functionally with relevant Stakeholders to define and document a Buy v. Build checklist that considers the following areas: (a) Is the scope of the Product manageable, given the results of the Organisation Capacity Analysis; (b) Can bought Models be used for other Products (eg. transfer learning); (c) Does the Organisation have the in-house expertise required to acquire and label the training data, given the Product Team Composition; (d) How much would it cost to acquire a properly labeled training dataset; (e) Given the Product Team Composition, does the Organisation have the in-house expertise required to retrain Models, if necessary; (f) How important is Model customization and, if so, can bought Models be customised; (g) Are the Acceptance Criteria - Accuracy, Bias, and Fairness requirements for bought Models feasible given the timeline, Product Team Composition, and Organisation Capacity Analysis; and/or (h) What are the usage limits and costs for pre-trained Models.	To (a) ensure that the Organisation's decision to either purchase or build the Models is appropriate based on Organisation capacity and/or constraints; and (b) highlight associated risks that might occur in the Product Lifecycle.
10.4.	POC-to-Production Go/No-Go Analysis	The Product Team should work cross-functionally with relevant Stakeholders to define and document a Go/No-Go checklist that considers qualitative and quantitative factors in the following areas: (a) Can POC-to-Production Checklist be adequately addressed; (b) Is the Product Cost Analysis still feasible; (c) Does the Product Team have approval for a Product maintenance budget; (d) Are the updates, upgrades, and add-ons to the data infrastructure near completion; (e) What is the state of customer process reconstruction and end-user training; (f) Has the failsafe, rollback, or emergency shutdown plan been completed and approved; and/or (g) Have the communication and mitigation plans in case of failsafe, rollback, or emergency shutdown been completed and approved.	To (a) ensure that the solution should be deployed in production and/or Product Domains; and (b) highlight associated risks that might occur in the Product Lifecycle.