Self-fulfilling Prophecies

From The Foundation for Best Practices in Machine Learning
Technical Best Practices > Systemic Stability > Self-fulfilling Prophecies

Self-fulfilling Prophecies

Control

Document and assess whether applying Product Outputs will result in change to Product inputs, dependencies and/or Domain(s) (other than those mentioned in controls elsewhere) and whether this is a self-reinforcing interaction. If true, attempt to mitigate associated effects through refining Product Output and/or Model design and/or development.


Aim

To (a) determine and prevent Product and/or Model risk in - (i) progressively strengthening biases (from encoded assumptions and definitions to datasets to algorithms chosen); (ii) progressively reinforcing Model errors and/or Product generalizations; (iii) progressively losing sensitivity to data and/or Domain changes; (iv) suffering from self-reinforcing and/or exponential run-away effects; (b) determine and prevent risks of unpredictable behaviour once the Product Outcomes are applied; and (c) highlight associated risks in the Product Lifecycle.


Additional Information