Hint

To view additional information and to make edit suggestions, click the individual items.

Systemic Stability

Objective

To prevent (in)direct adverse social and environmental effects as a consequence of interactions amongst Products, Models, the Organisation, and the Public.

20.1. Product Definition(s)

Objective

To investigate and mitigate unforeseen social and environmental chain effects and/or risks caused through Product Definition(s).

Item nr.	Item Name and Page	Control	Aim
20.1.1.	Product Assumption Susceptibility	Document and assess whether applying Product Outputs will result in invalidating Product Assumptions. If so, attempt to redefine Product Assumptions to warrant their longevity.	To (a) prevent unpredictable social and/or environmental behaviour through Product Outcomes; and (b) highlight associated risks in the Product Lifecycle.

20.2. Exploration

Objective

To investigate and mitigate unforeseen social and environmental chain effects and/or risks caused through Product exploration.

Item nr.	Item Name and Page	Control	Aim
20.2.1.	Selection Function Susceptibility	Document and assess whether applying Product Outputs will result in changes to the Selection Function, and whether this is a self-reinforcing interaction. If true, attempt to mitigate or stabilize associated effects through refining Product Definition(s) and/or improving Model design and/or Product and process implementation.	To (a) determine and prevent Product and/or Model risk in - (i) progressively strengthening biases (from encoded assumptions and definitions to datasets to algorithms chosen); (ii) progressively reinforcing Model errors and/or Product generalizations; (iii) progressively losing sensitivity to data and/or Domain changes; (iv) suffering from self-reinforcing and/or exponential run-away effects; (b) determine and prevent risks of unpredictable behaviour once the Product Outcomes are applied; and (c) highlight associated risks in the Product Lifecycle.
20.2.2.	Data Definition Susceptibility	Document and assess whether applying Product Outputs will result in changes to the Product data definitions, and whether this is a self-reinforcing interaction. If true, attempt to mitigate or stabilize associated effects through refining Product Definition(s) and/or improving Model design and/or Product and process implementation.	To (a) determine and prevent Product and/or Model risk in - (i) progressively strengthening biases (from encoded assumptions and definitions to datasets to algorithms chosen); (ii) progressively reinforcing Model errors and/or Product generalizations; (iii) progressively losing sensitivity to data and/or Domain changes; (iv) suffering from self-reinforcing and/or exponential run-away effects; (b) determine and prevent risks of unpredictable behaviour once the Product Outcomes are applied; and (c) highlight associated risks in the Product Lifecycle.
20.2.3.	Data Generating Process Susceptibility	Document and assess whether applying Product Outputs will result in changes to the data generating process, and whether this is a self-reinforcing interaction. If true, attempt to mitigate or stabilize associated effects through refining Product Definition(s) and/or improving Model design and/or Product and process implementation.	To (a) determine and prevent Product and/or Model risk in - (i) progressively strengthening biases (from encoded assumptions and definitions to datasets to algorithms chosen); (ii) progressively reinforcing Model errors and/or Product generalizations; (iii) progressively losing sensitivity to data and/or Domain changes; (iv) suffering from self-reinforcing and/or exponential run-away effects; (b) determine and prevent risks of unpredictable behaviour once the Product Outcomes are applied; and (c) highlight associated risks in the Product Lifecycle.
20.2.4.	Data Distributions Susceptibility	Document and assess whether applying Product Outputs will result in changes to the data distributions, and whether this is a self-reinforcing interaction. If true, attempt to mitigate or stabilize associated effects through refining Product Definition(s) and/or improving Model design and/or Product and process implementation.	To (a) determine and prevent Product and/or Model risk in - (i) progressively strengthening biases (from encoded assumptions and definitions to datasets to algorithms chosen); (ii) progressively reinforcing Model errors and/or Product generalizations; (iii) progressively losing sensitivity to data and/or Domain changes; (iv) suffering from self-reinforcing and/or exponential run-away effects; (b) determine and prevent risks of unpredictable behaviour once the Product Outcomes are applied; and (c) highlight associated risks in the Product Lifecycle.
20.2.5.	Hidden Variable Susceptibility	Document and assess whether applying Product Outputs will result in the creation of new hidden Variables in the system. If true, record the new Variable during data gathering, or prevent the creation of the new Variable through improved Product Definition(s) and implementation.	To (a) determine and prevent risk of unpredictable behaviour once the Product Outcomes are applied; and (b) highlight associated risks in the Product Lifecycle.

20.3. Development

Objective

To investigate and mitigate unforeseen social and environmental chain effects and/or risks caused through Product development.

Item nr.	Item Name and Page	Control	Aim
20.3.1.	Target Feature Definition Susceptibility	Document and assess whether applying Product Outputs will result in changes to the Target Feature definition. If true, attempt to mitigate associated effects through refining Product Output and/or Model design and/or development.	To (a) determine and prevent risk of unpredictable behaviour once the Product outcomes are applied; and (b) highlight associated risks in the Product Lifecycle.
20.3.2.	Optimization Feedback Loop Susceptibility	Document and assess whether the cost function and/or optimization algorithm exhibits a feedback loop behaviour that includes the gathering of data that has been influenced by previous Model iterations, and whether this behaviour is self-reinforcing or self-limiting. If true, attempt to mitigate associated effects through refining Product Output and/or Model design and/or development.	To (a) determine and prevent Product and/or Model risk in - (i) progressively strengthening biases (from encoded assumptions and definitions to datasets to algorithms chosen); (ii) progressively reinforcing Model errors and/or Product generalizations; (iii) progressively losing sensitivity to data and/or Domain changes; (iv) suffering from self-reinforcing and/or exponential run-away effects; (b) determine and prevent risks of unpredictable behaviour once the Product Outcomes are applied; and (c) highlight associated risks in the Product Lifecycle.

20.4. Production

Objective

To investigate and mitigate unforeseen social and environmental chain effects and/or risks caused through Product application.

Item nr.	Item Name and Page	Control	Aim
20.4.1.	Self-fulfilling Prophecies	Document and assess whether applying Product Outputs will result in change to Product inputs, dependencies and/or Domain(s) (other than those mentioned in controls elsewhere) and whether this is a self-reinforcing interaction. If true, attempt to mitigate associated effects through refining Product Output and/or Model design and/or development.	To (a) determine and prevent Product and/or Model risk in - (i) progressively strengthening biases (from encoded assumptions and definitions to datasets to algorithms chosen); (ii) progressively reinforcing Model errors and/or Product generalizations; (iii) progressively losing sensitivity to data and/or Domain changes; (iv) suffering from self-reinforcing and/or exponential run-away effects; (b) determine and prevent risks of unpredictable behaviour once the Product Outcomes are applied; and (c) highlight associated risks in the Product Lifecycle.
20.4.2.	Hidden Variable Dependencies	Document and assess whether the effect of applying Product Outputs depends on Hidden Variables. If true, control for Hidden Variables, for example through marginalization and/or by deriving indicators for Hidden Variables influence.	To (a) prevent diverging effects on seemingly similar individuals or datapoints; (b) prevent or detect high-risk interactions; and (c) highlight associated risks in the Product Lifecycle.
20.4.3.	Society Susceptibility	Document and assess whether applying Product Outputs results in potentially harmful societal or environmental changes, and research the possible knock-on effects as far as reasonably practical.	To (a) identify and prevent both direct and indirect adverse effects on society and the environment; (b) determine if there is a risk of unpredictable behaviour once the Product Outcomes are applied; and (c) highlight associated risks in the Product Lifecycle.
20.4.4.	Domain Susceptibility	Document and assess whether applying Product Outputs results in changes to application Domain(s), and research the possible knock-on effects as far as reasonably practical.	To (a) identify and prevent both direct and indirect adverse effects on Product Domain(s); (b) determine if there is a risk of unpredictable behaviour once the Product Outcomes are applied; and (c) highlight associated risks in the Product Lifecycle.
20.4.5.	Other Organisation Products Susceptibility	Document and assess whether applying Product Outputs result in changes to inputs, dependencies and/or context for other Organisation Products. If true, attempt to mitigate associated effects through refining Product Output and/or Model design and/or development.	To (a) identify and prevent both direct and indirect adverse effects on the Organisation or other Organisation Products; (b) determine if there is a risk of unpredictable behaviour once the Product Outcomes are applied; and (c) highlight associated risks in the Product Lifecycle.