Data Generating Process Temporal Stability

From The Foundation for Best Practices in Machine Learning
Technical Best Practices > Performance Robustness > Data Generating Process Temporal Stability

Data Generating Process Temporal Stability

Control

Document and assess the historic and prospective behaviour of data generating processes, and their influence on the Selection Function. If unstable, take measures to account for past and future changes and/or promote the stability and consistency of data generation processes as much as is reasonably practical.


Aim

To (a) assess and control for hard-to-measure changes to the relation between Model datasets and Product Domain(s); (b) identify the risk of hard-to-diagnose Model performance degradation and bias throughout Product Lifecycle (to be controlled by 14.3.6. - Model Drift & Model Robustness Simulations); and (c) highlight associated risks that might occur in the Product Lifecycle


Additional Information