Fairness Testing of Outcomes

From The Foundation for Best Practices in Machine Learning

Fairness Testing of Outcomes

Control

Focus fairness testing initially on outcomes that are immediately experienced by (Sub)populations. For example, if a model uses a series of sub-Models to generate a score and a threshold is applied to that score to determine an Outcome, focus on Fairness issues related to that Outcome. If issues are identified, then diagnose the issue by moving "up-the-chain" and testing the Model score and sub-Models.


Aim

To (a) ensure that the testing performed best reflects what will happen when Models are deployed in the real world; and (b) highlight associated risks that might occur in the Product Lifecycle.


Additional Information