Product and Model Oversight & Management
Product and Model Oversight & Management
5.1. Fairness & Non-Discrimination
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.1.1. | Fairness Policy |
A Policy and Guide, which promotes Product Fairness in - (a) Product Definitions and Product design, development, and implementation; (b) data processing; and (c) Model design, development, training, and output ought to be derived by Data Science Managers and approved by the Managerial Committee. |
To ensure the Fairness of Machine Learning. |
5.1.2. | Product Fairness Procedures |
A set of Procedures to operationalise the Fairness Policy should be developed and implemented within Products in light of Product Definitions, the Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure the Fairness of Products and Models. |
5.1.3. | Fairness Assessments |
Products should regularly complete Fairness assessments according to Product Lifecycle and Workflow Descriptions to the extent that is reasonably practical. Assessment findings ought to be documented by the Product Team and reviewed by Data Science Managers and, when relevant, the Management Committee. |
To analyse, test, and report the risks identified with, and measures taken to ensure, Product Fairness and non-discrimination at regular intervals within the Product Lifecycle and Workflow Description. |
5.1.4. | Review of the Fairness Policy |
The Fairness Policy should be reviewed periodically, or if significant changes occur, by Data Science Managers to ensure its continued effectiveness, suitability, and accuracy. |
To ensure that the Fairness Policy is kept up-to-date. |
5.1.5. | Review of Product Fairness Procedures |
Product Fairness Procedures should be reviewed periodically, or if significant changes occur, by the Product Team to ensure their continued effectiveness, suitability, and accuracy. |
To ensure that Product Fairness Procedures are kept up-to-date. |
5.2. Data Quality
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.2.1. | Data Quality Policy |
A Policy and Guide, which describes the assessment and remediation of an Organisation's Data Quality. Chapters relating to (a) Product Definitions and Product design, development, and implementation; (b) data processing; and (c) Model design, development, training, and output ought to be derived and included by Data Science Managers and approved by the Managerial Committee. |
To ensure the Data Quality of Machine Learning. |
5.2.2. | Product Data Quality Procedures |
A set of Procedures to operationalise the Data Quality Policy should be developed and implemented within Products in light of Product Definitions, the Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure the Data Quality of Products and Models. |
5.2.3. | Data Quality Assessments |
Products should regularly complete Data Quality assessments according to Product Lifecycle and Workflow Descriptions to the extent that is reasonably practical. Assessment findings ought to be documented by the Product Team and reviewed by Data Science Managers and, when relevant, the Management Committee. |
To analyse, test, and report the risks identified with, and measures taken to ensure, Product Data Quality at regular intervals within the Product Lifecycle and Workflow Description. |
5.2.4. | Review of the Data Quality Policy |
The Data Quality Policy should be reviewed periodically, or if significant changes occur, by Data Science Managers to ensure its continued effectiveness, suitability, and accuracy. |
To ensure that the Data Quality Policy is kept up-to-date. |
5.2.5. | Review of Product Data Quality Procedures |
Product Data Quality Procedures should be reviewed periodically, or if significant changes occur, by the Product Team to ensure their continued effectiveness, suitability, and accuracy. |
To ensure that Product Data Quality Procedures are kept up-to-date. |
5.3. Representativeness & Specification
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.3.1. | Representativeness & Specification Policy |
A Policy and Guide, which promotes Representativeness and Specification in - (a) Product Definitions and Product design, development, and implementation; (b) data processing; and (c) Model design, development, training, and output ought to be derived by Data Science Managers and approved by the Managerial Committee. |
To ensure the Representativeness and Specification of Machine Learning. |
5.3.2. | Product Representativeness & Specification Procedures |
A set of Procedures to operationalise the Representativeness & Specification Policy should be developed and implemented within Products in light of Product Definitions, the Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure the Representativeness and Specification of Products and Models. |
5.3.3. | Representativeness & Specification Assessments |
Products should regularly complete Representativeness & Specification assessments according to Product Lifecycle and Workflow Descriptions to the extent that is reasonably practical. Assessment findings ought to be documented by the Product Team and reviewed by Data Science Managers and, when relevant, the Management Committee. |
To analyse, test, and report the risks identified with, and measures taken to ensure, Product Representativeness and Specification at regular intervals within the Product Lifecycle and Workflow Description. |
5.3.4. | Review of the Representativeness & Specification Policy |
The Representativeness & Specification Policy should be reviewed periodically, or if significant changes occur, by Data Science Managers to ensure its continued effectiveness, suitability, and accuracy. |
To ensure that the Representativeness & Specification Policy is kept up-to-date. |
5.3.5. | Review of Product Representativeness & Specification Procedures |
Product Representativeness & Specification Procedures should be reviewed periodically, or if significant changes occur, by the Product Team to ensure their continued effectiveness, suitability, and accuracy. |
To ensure that Representativeness & Specification Procedures are kept up-to-date. |
5.4. Performance Robustness
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.4.1. | Performance Robustness Policy |
A Policy and Guide, which promotes Product Performance Robustness in - (a) Product Definitions and Product design, development, and implementation; (b) data processing; and (c) Model design, development, training, and output ought to be derived by Data Science Managers and approved by the Managerial Committee. |
To ensure the Performance and Robustness of Machine Learning. |
5.4.2. | Product Performance Robustness Procedures |
A set of Procedures to operationalise the Performance Robustness Policy should be developed and implemented within Products in light of Product Definitions, the Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions . |
To ensure the Performance Robustness of Products and Models. |
5.4.3. | Performance Robustness Assessments |
Products should regularly complete Performance Robustness Assessments according to Product Lifecycle and Workflow Descriptions to the extent that is reasonably practical. Assessment findings ought to be documented by the Product Team and reviewed by Data Science Managers and, when relevant, the Management Committee. |
To analyse, test, and report the risks identified with, and measures taken to ensure, Product Performance Robustness at regular intervals within the Product Lifecycle and Workflow Description. |
5.4.4. | Review of the Performance Robustness Policy |
The Performance Robustness Policy should be reviewed periodically, or if significant changes occur, by Data Science Managers to ensure its continued effectiveness, suitability, and accuracy. |
To ensure that the Performance Robustness Policy is kept up-to-date. |
5.4.5. | Review of Product Performance Robustness Procedures |
Product Performance Robustness Procedures should be reviewed periodically, or if significant changes occur, by the Product Team to ensure their continued effectiveness, suitability, and accuracy. |
To ensure that Performance Robustness Procedures are kept up-to-date. |
5.5. Monitoring & Maintenance
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.5.1. | Monitoring & Maintenance Policy |
A Policy and Guide, which promotes Product monitoring and maintenance in - (a) Product Definitions and Product design, development, and implementation; (b) data processing; and (c) Model design, development, training, and output ought to be derived by Data Science Managers and approved by the Managerial Committee. |
To ensure the monitoring and maintenance of Machine Learning. |
5.5.2. | Product Monitoring & Maintenance Procedures |
A set of Procedures to operationalise the Monitoring & Maintenance Policy should be developed and implemented within Products in light of Product Definitions, the Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure the monitoring and maintenance of Products and Models. |
5.5.3. | Monitoring & Maintenance Assessments |
Products should regularly complete Monitoring & Maintenance assessments according to Product Lifecycle and Workflow Descriptions to the extent that is reasonably practical. Assessment findings ought to be documented by the Product Team and reviewed by Data Science Managers and, when relevant, the Management Committee. |
To analyse, test, and report the risks identified with, and measures taken to ensure, Product monitoring and maintenance at regular intervals within the Product Lifecycle and Workflow Description. |
5.5.4. | Review of the Monitoring & Maintenance Policy |
The Monitoring & Maintenance Policy should be reviewed periodically, or if significant changes occur, by Data Science Managers to ensure its continued effectiveness, suitability, and accuracy. |
To ensure that the Monitoring & Maintenance Policy is kept up-to-date. |
5.5.5. | Review of Product Monitoring & Maintenance Procedures |
Product Monitoring & Maintenance Procedures should be reviewed periodically, or if significant changes occur, by the Product Team to ensure their continued effectiveness, suitability, and accuracy. |
To ensure that Monitoring & Maintenance Procedures are kept up-to-date. |
5.6. Explainability
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.6.1. | Explainability Policy |
A Policy and Guide, which promotes Product Explainability in - (a) Product Definitions and Product design, development, and implementation; (b) data processing; and (c) Model design, development, training, and output ought to be derived by Data Science Managers and approved by the Managerial Committee. |
To ensure the Explainability of Machine Learning. |
5.6.2. | Product Explainability Procedures |
A set of Procedures to operationalise the Explainability Policy should be developed and implemented within Products in light of Product Definitions, the Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure the Explainability of Products and Models. |
5.6.3. | Explainability Assessments |
Products should regularly complete Explainability assessments according to Product Lifecycle and Workflow Descriptions to the extent that is reasonably practical. Assessment findings ought to be documented by the Product Team and reviewed by Data Science Managers and, when relevant, the Management Committee. |
To analyse, test, and report the risks identified with, and measures taken to ensure, Product Explainability at regular intervals within the Product Lifecycle and Workflow Description. |
5.6.4. | Review of the Explainability Policy |
The Explainability Policy should be reviewed periodically, or if significant changes occur, by Data Science Managers to ensure its continued effectiveness, suitability, and accuracy. |
To ensure that the Explainability Policy is kept up-to-date. |
5.6.5. | Review of Product Explainability Procedures |
Product Explainability Procedures should be reviewed periodically, or if significant changes occur, by the Product Team to ensure their continued effectiveness, suitability, and accuracy. |
To ensure that Product Explainability Procedures are kept up-to-date. |
5.7. Safety & Security
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.7.1. | Safety & Security Policy |
A Policy and Guide, which promotes Product Safety & Security in - (a) Product Definitions and Product design, development, and implementation; (b) data processing; and (c) Model design, development, training, and output ought to be derived by Data Science Managers and approved by the Managerial Committee. |
To ensure the Safety of Machine Learning. |
5.7.2. | Product Safety Procedures |
A set of Procedures to operationalise the Safety Policy should be developed and implemented within Products in light of Product Definitions, the Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions . |
To ensure the Safety of Products and Models. |
5.7.3. | Safety Assessments |
Products should regularly complete Safety assessments according to Product Lifecycle and Workflow Descriptions to the extent that is reasonably practical. Assessment findings ought to be documented by the Product Team and reviewed by Data Science Managers and, when relevant, the Management Committee. |
To analyse, test, and report the risks identified with, and measures taken to ensure, Product Safety at regular intervals within the Product Lifecycle and Workflow Description. |
5.7.4. | Review of the Safety Policy |
The Safety Policy should be reviewed periodically, or if significant changes occur, by Data Science Managers to ensure its continued effectiveness, suitability, and accuracy. |
To ensure that the Safety Policy is kept up-to-date. |
5.7.5. | Review of Product Safety Procedures |
Product Safety Procedures should be reviewed periodically, or if significant changes occur, by the Product Team to ensure their continued effectiveness, suitability, and accuracy. |
To ensure that Product Safety Procedures are kept up-to-date. |
5.8. Human-Centric Design & Redress
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.8.1. | Human-centric Design & Redress Policy |
A Policy and Guide, which promotes Product Human-Centric Design & Redress in - (a) Product Definitions and Product design, development, and implementation; (b) data processing; and (c) Model design, development, training, and output ought to be derived by Data Science Managers and approved by the Managerial Committee. |
To ensure the Human-Centric Design & Redress of Machine Learning. |
5.8.2. | Product Human-Centric Design & Redress Procedures |
A set of Procedures to operationalise the Human-Centric Design & Redress Policy should be developed and implemented within Products in light of Product Definitions, the Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure the Human-Centric Design & Redress of Products and Models. |
5.8.3. | Human-Centric Design & Redress Assessments |
Products should regularly complete Human-Centric Design & Redress assessments according to Product Lifecycle and Workflow Descriptions to the extent that is reasonably practical. Assessment findings ought to be documented by the Product Team and reviewed by Data Science Managers and, when relevant, the Management Committee. |
To analyse, test, and report the risks identified with, and measures taken to ensure, Product Human-Centric Design & Redress at regular intervals within the Product Lifecycle and Workflow Description. |
5.8.4. | Review of the Human-Centric Design & Redress Policy |
The Human-Centric Design & Redress Policy should be reviewed periodically, or if significant changes occur, by Data Science Managers to ensure its continued effectiveness, suitability, and accuracy. |
To ensure that the Human-Centric Design & Redress Policy is kept up-to-date. |
5.8.5. | Review of Product Human-Centric Design & Redress Procedures |
Product Human-Centric Design & Redress Procedures should be reviewed periodically, or if significant changes occur, by the Product Team to ensure their continued effectiveness, suitability, and accuracy. |
To ensure that Product Human-Centric Design & Redress Procedures are kept up-to-date. |
5.9. Systemic Stability
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.9.1. | Systemic Stability Policy |
A Policy and Guide, which promotes awareness and control of systemic effects and interactions in - (a) Product Definitions and Product design, development, and implementation; (b) data processing; and (c) Model design, development, training, and output ought to be derived by Data Science Managers and approved by the Managerial Committee. |
To ensure the Systemic Stability of Machine Learning. |
5.9.2. | Product Systemic Stability Procedures |
A set of Procedures to operationalise the Systemic Stability Policy should be developed and implemented within Products in light of Product Definitions, the Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure the Systemic Stability of Products. |
5.9.3. | Systemic Stability Assessments |
Products should regularly complete Systemic Stability according to Product Lifecycle and Workflow Descriptions to the extent that is reasonably practical. Assessment findings ought to be documented by the Product Team and reviewed by Data Science Managers and, when relevant, the Management Committee. |
To analyse, test, and report the risks identified with, and measures taken to ensure, Product Systemic Stability at regular intervals within the Product Lifecycle and Workflow Description. |
5.9.4. | Review of the Systemic Stability Policy |
Systemic Stability should be reviewed periodically, or if significant changes occur, by Data Science Managers to ensure its continued effectiveness, suitability, and accuracy. |
To ensure that the Systemic Stability Policy is kept up-to-date. |
5.9.5. | Review of Systemic Stability Procedures |
Product Systemic Stability should be reviewed periodically, or if significant changes occur, by the Product Team to ensure their continued effectiveness, suitability, and accuracy. |
To ensure that Product Systemic Stability Procedures are kept up-to-date. |
5.10. Product Traceability
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.10.1. | Product Traceability Policy |
A Policy and Guide, which promotes Product Traceability during and after - (a) Product Definitions and Product design, development, and implementation; (b) data processing; and (c) Model design, development, training, and output ought to be derived by Data Science Managers and approved by the Managerial Committee. |
To ensure the Product Traceability of Machine Learning. |
5.10.2. | Product Traceability Procedures |
A set of Procedures to operationalise the Traceability Policy should be developed and implemented within Products in light of Product Definitions, the Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure the Product Traceability of Products. |
5.10.3. | Review of the Product Traceability Policy |
The Product Traceability should be reviewed periodically, or if significant changes occur, by Data Science Managers to ensure its continued effectiveness, suitability, and accuracy. |
To ensure that the Product Traceability Policy is kept up-to-date. |
5.10.4. | Review of Product Traceability Procedures |
Product Product Traceability should be reviewed periodically, or if significant changes occur, by the Product Team to ensure their continued effectiveness, suitability, and accuracy. |
To ensure that Product Product Traceability Procedures are kept up-to-date. |
5.11. Product Decision-Making
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.11.1. | Product Decision-Making Policy |
A Policy which promotes Product decision-making clarity in - (a) Product Definitions and Product design, development, and implementation; (b) data processing; (c) Model design, development, training, and output; and (d) consideration for the Product Risk Classification Portfolio ought to be derived by Data Science Managers and approved by the Managerial Committee. |
To ensure Product decisions - (a) are based on all available information, inclusive of those derived from Policies and Procedures of this document; (b) follow from and are aligned with the Product Lifecycle and Workflow Description; and (c) are made with reasonable care to avoid cognitive bias. |
5.11.2. | Product Decision-Making Procedures |
A set of Procedures to operationalise the Decision-Making Policy within Products should be developed and implemented, inclusive of consultations with Business Stakeholders. |
To ensure clear, informed, unbiased and collaborative decision-making in Products. |
5.11.3. | Product Decision-Making Diversity |
A diversity of Organisation stakeholder and Business Stakeholder opinions and input should be obtained, considered and, when relevant, weighed when making material Product decisions |
To ensure a diversity of opinions when making material Product decisions. |
5.11.4. | Product Decision-Making Documentation |
Product decisions should be documented, indexed, stored and, when relevant, reviewed. |
To ensure that Product decisions are documented and indexed to warrant their effective management and oversight. |
5.12. Product Capabilities
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.12.1. | Financial Resource Allocation |
Sufficient financial resources ought to be allocated to Products based on their Product Definitions, Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure that Products have sufficient financial resources to allow for their success. |
5.12.2. | Human Resource Allocation |
Sufficient human resources ought to be allocated to Products based on their Product Definitions, Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure that Products have sufficient human resources to allow for their success. |
5.12.3. | Assets Allocation |
Sufficient Assets ought to be allocated to Products based on their Product Definitions, Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure that Products have sufficient Assets to allow for their success. |
5.12.4. | Software Allocation |
Sufficient Software ought to be allocated to Products based on their Product Definitions, Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure that Products have sufficient Software to allow for their success. |
5.12.5. | Knowledge & Development |
Product Teams should receive sufficient training and development based on the Product Definitions, Product Risk Classification Portfolio, and the Product Lifecycle and Workflow Descriptions. |
To ensure that Product Teams have sufficient training and development to allow for their success. |
5.12.6. | Review of Product Capabilities |
Product resource allocation ought to be periodically reviewed, or if significant changes occur, by Data Science Managers, in consultation with Product Owners, to ensure their continued effectiveness, suitability, and accuracy. |
To ensure that Products resources are sufficiently maintained. |
5.13. Product Record
Item nr. | Item Name and Page | Control | Aim |
---|---|---|---|
5.13.1. | Product Record |
A clear and detailed Product record should be continually kept of Product design, development, implementation, deliverables, progress and employee tasks. |
To ensure that a clear Product record is kept to warrant effective oversight, management and accountability. |
5.13.2. | Employee and Product Team Documentation |
Employee and Product Team processes ought to promote the documentation of Product tasks, discussions and deliverables when relevant and as much as is reasonably practical. |
To ensure that sufficient documentation is kept to warrant effective oversight, management and accountability. |
5.13.3. | Log of User Access |
A formal log of user access rights to Products should be maintained and reviewed at regular intervals by Product Owners. |
To ensure the management, integrity and review of user access. |
5.13.4. | Event Logs |
Event logs of Product user activities, exceptions, and faults should be produced, kept and regularly reviewed by Product Owners. |
To formally index and manage user activities to maintain Product oversight and integrity. |