Hint

To view additional information and to make edit suggestions, click the individual items.

Testing

Objective

To ensure (a) that robust, effective, and efficient strategies, methodologies and schedules are developed for testing the Product; and (b) clear maintenance metrics and/or phases to warrant continued Product alignment with chosen metrics and performance goals.

Item nr.	Item Name and Page	Control	Aim
8.1.	Testing Design and Scheduling Framework	Document and define a testing design and schedule that does not artificially constrain the testing process. Incorporate the Feedback Loop Analysis in the testing design. Review the Automation Analysis in determining what level of automation is appropriate for each stage of testing. Ensure the individuals chosen through the Testing Participant Identification process are involved at the earliest stages of the testing schedule as practical. Post Product deployment, document and define a framework and process for testing and selecting variations of the production Model.	To (a) ensure a robust and feasible testing design and scheduling framework that allows for effective Product optimization; and (b) highlight associated risks that might occur in the Product Lifecycle.
8.2.	Testing Participant Identification	Document and define a process for identifying test participants as required by User Experience Mapping and Societal Context, as discussed in Section 4 - Problem Mapping; and Section 3 - Context. Determine a framework for ensuring that testing participants are intentionally diverse across use cases, user types and roles, and internal and external Stakeholders.	To (a) ensure testing for user impact and participant pool diversity; and (b) highlight associated risks that might occur in the Product Lifecycle.
8.3.	Automation Analysis	Determine and define data, Model, and component integration validations that can be reasonably automated. Assess and define any processes during the development, deployment, or maintenance phases that could benefit from integrating automation into the testing infrastructure. Be sure to review - (a) the Organisation Capacity Analysis, as discussed in Section 4 - Problem Mapping, while determining the feasibility of automating the identified processes; and/or (b) the Industry, Deployment, and Societal Contexts, as discussed in Section 3 - Context, to uncover any gaps or misalignment raised by the automation of any identified process.	To (a) identify suitable and effective areas for incorporating testing automation within the Product development, deployment, and maintenance phases; and (b) highlight associated risks that might occur in the Product Lifecycle.
8.4.	Feedback Loop	Document and define a feedback loop that enables monitoring of stability, performance, and operations metrics, and counter-metrics, as required by Performance Robustness, Monitoring and Maintenance, and Systemic Stability, as discussed more thoroughly in Section 14 - Performance Robustness; Section 15 - Monitoring & Maintenance; and Section 20 - Systemic Stability. Develop and incorporate a method for flagging bias and for issue reporting. Document and define a process for real-time sharing of testing participant feedback with the development and maintenance teams. Incorporate the Feedback Loop in the Testing Design and Scheduling Framework to ensure that the Features the Model is utilizing are acceptable for the application during the development, deployment, and maintenance phases.	To (a) ensure robust and responsive feedback loop measures that enable monitoring of necessary metrics and effectively integrate into the Testing Design and Scheduling Framework; and (b) highlight associated risks that might occur in the Product Lifecycle.