A Bandit-based Algorithm for Fairness-Aware Hyperparameter Optimization

This repository contains ML artifacts and other materials from the experiments performed on the paper.

Key Contributions

Repository Structure

Fairband: Selected Fairness-Accuracy Trade-off, discriminated by Model Type

EG Experiment

Running Fairband (15 runs) on the Adult dataset supplied with the following model choices: Neural Network (NN), Random Forest (RF), Decision Tree (DT), Logistic Regression (LR), LightGBM (LGBM), and Exponentiated Gradient reduction for fair classification (EG).

EG is a state-of-the-art bias reduction method available at fairlearn.

As shown by the plot, blindly applying bias reduction techniques may lead to suboptimal fairness-accuracy trade-offs. In this example, EG can be dominated by NN models. Fairband should be used in conjunction with a wide portfolio of model choices.

Citing

@misc{cruz2020banditbased,
      title={A Bandit-Based Algorithm for Fairness-Aware Hyperparameter Optimization}, 
      author={F. Cruz, Andr{\'e} and Saleiro, Pedro and Bel{\'e}m, Catarina and Soares, Carlos and Bizarro, Pedro},
      year={2020},
      eprint={2010.03665},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}