ReSGrAL: Fairness-Sensitive Active Learning

The use of machine learning models for decision support in public organizations is generally constrained by limited labeled data and the high cost of labeling. Additionally, models used by the public sector have been shown to express various biases (e.g., towards gender or ethnicity), highlighting the urgency to address fairness concerns. Although active learning has proven to be useful in efficiently selecting instances for labeling (and thus reducing the impact of the first issue), its impact on fairness is still unclear. The present work has a two-fold objective. First, it aims to experimentally study the relationship between active learning and fairness. Second, it explores fairness-sensitive methods for active learning, proposing two novel variations, Representative SubGroup Active Learning (ReSGrAL) and Fair ReSGrAL. Our experiments show that, in general, active learning can increase model unfairness beyond the dataset bias, and thus caution is needed when using active learning in sensitive contexts. Fortunately, we also show that techniques like ReSGrAL can mitigate unfairness without sacrificing accuracy.