Selection Function Temporal Stability

From The Foundation for Best Practices in Machine Learning
Technical Best Practices > Performance Robustness > Selection Function Temporal Stability

Selection Function Temporal Stability

Control

Document and assess the historic and prospective behaviour of Selection Function(s) of Model data. (See Section 13.2.4. - Selection Function for more information.) If unstable, take measures to account for past and future changes, and/or promote the consistency and representativeness of Model datasets and data gathering as much as is reasonably practical.


Aim

To (a) assess and control for hard-to-measure changes to the relation between Model datasets and Product Domain(s); (b) identify the risk of hard-to-diagnose Model performance degradation and bias throughout Product Lifecycle (to be controlled by 14.3.6. - Model Drift & Model Robustness Simulations); and (c) highlight associated risks that might occur in the Product Lifecycle.


Additional Information