Adaptive Runtime Fairness Monitoring for Credit Scoring During Economic Fluctuations

Abstract

The need for fair automated decision making is increasing as algorithms continue to have a growing impact on humans. Runtime fairness monitors are algorithms that detect fairness violations of fairness constraints as an algorithm is being run on real-world data, through computing statistics over observed data instances. These monitors however often assume that the input data comes from a static distribution. Our research aims to build a runtime fairness monitor for credit scoring algorithms that tracks distribution shifts in the underlying economic state to detect fairness violations more quickly. To accomplish this, we have designed a synthetic credit scoring dataset that simulates economic fluctuations. Our main contribution is the development of an adaptive fairness monitoring algorithm that dynamically adjusts to economic fluctuations, through a sliding window that disregards older, less representative samples when fluctuations in the underlying distribution our detected. We have tested our adaptive monitor against a baseline monitor that simply computes fairness metrics over all observed data instances. Our preliminary results show that our adaptive approach improves the speed of detecting fairness violations compared to the more traditional monitoring method. In conclusion, there is potential for incorporating economic state detection into credit score fairness monitoring. Future work should validate these findings with real-world data and explore additional fairness metrics.