Optimization Feedback Loop Susceptibility

From The Foundation for Best Practices in Machine Learning
Technical Best Practices > Systemic Stability > Optimization Feedback Loop Susceptibility

Optimization Feedback Loop Susceptibility

Control

Document and assess whether the cost function and/or optimization algorithm exhibits a feedback loop behaviour that includes the gathering of data that has been influenced by previous Model iterations, and whether this behaviour is self-reinforcing or self-limiting. If true, attempt to mitigate associated effects through refining Product Output and/or Model design and/or development.


Aim

To (a) determine and prevent Product and/or Model risk in - (i) progressively strengthening biases (from encoded assumptions and definitions to datasets to algorithms chosen); (ii) progressively reinforcing Model errors and/or Product generalizations; (iii) progressively losing sensitivity to data and/or Domain changes; (iv) suffering from self-reinforcing and/or exponential run-away effects; (b) determine and prevent risks of unpredictable behaviour once the Product Outcomes are applied; and (c) highlight associated risks in the Product Lifecycle.


Additional Information