Toolkit Bias Auditor

Dataset Bias Auditor

Upload a CSV dataset or use a sample to analyze bias and representation imbalances.

Same score, unequal mistakes

The recidivism sample's risk score never sees race. Yet among people who did not go on to reoffend, it flags Black defendants as high risk far more often than White defendants. Selection rates hide this; you need the confusion matrix, which needs ground truth.

Work one row out by hand first: count the people right of the line, then the shaded ones among them. Then load the Recidivism sample below and check the auditor gets the same number.

From data/recidivism-sample.csv (synthetic). Flagged = PredictedRisk High. * fewer than 25 people did not reoffend, too few to compare.

Data Source

Audit Configuration

Which outcome benefits the person. This is a value judgement, not a property of the data — you have to make it.