"If it didn't happen, why would I change my decision?": How Judges Respond to Counterfactual Explanations for the Public Safety Assessment

Researchers and policymakers are interested in algorithmic explanations as a mechanism for enabling more fair and responsible decision-making. In this study, we shed light on how judges interpret and respond to algorithmic explanations in the context of pretrial risk assessment instruments (PRAI). We found that, at first, all judges misinterpreted the counterfactuals in the explanations as real -- rather than hypothetical -- changes to defendants' criminal history profiles. Once judges understood the counterfactuals, they ignored them, preferring to make decisions based solely on the actual details of the defendant in question. Our findings suggest that using (at least this kind of) explanations to improve human and AI collaboration is not straightforward.