Institutions Make Mistakes: Which Should We Seek to Prevent; Which Ought We Fix After the Fact?

In experiments, people endorsed seemingly harsh policies — only to reverse course after the fact

Even the best thought-out policies can yield troubling errors. Some criminals fly under the radar; some innocents are convicted and go to prison. Some people entitled to health insurance coverage are denied their benefits; others filing fraudulent claims get paid.

UCLA Anderson’s Eitan Rude, a Ph.D. student, and Franklin Shaddy explore whether people think it’s more important to prevent such mistakes from happening in the first place or to fix them after the fact. In research published in Psychological Science, they present a complex answer that suggests it depends on timing — has the mistake already occurred or is it just theoretical? — and whether the mistake pertains to a system intended to punish or reward people.

Making a Trade-off

Across six experiments involving nearly 3,500 participants, Rude and Shaddy explore scenarios that involve a trade-off between two general types of outcomes:

• Those who don’t deserve to be punished or rewarded, but who are (defined as false positives).
• Those who deserve to be punished or rewarded, but who aren’t (false negatives).

They find that — depending on the context — acceptance of certain instances of these mistakes is greater when presented as a theoretical possibility as compared to when presented as an actual outcome.

Specifically, when the mistake involves punishments, we care more about preventing people from evading deserved punishment (false negatives) than we do about fixing these types of mistakes after the fact. However, this comes at the expense of addressing cases in which people who don’t deserve to be punished get a slap on the wrist by mistake (false positives). While we overwhelmingly support fixing these mistakes after the fact, we pay less attention to opportunities to prevent them from happening in advance.

For rewards, the opposite pattern holds. We care more about preventing undeserving people from reaping rewards (false positives), than we do about fixing such mistakes after the fact. And we care more about fixing cases in which benefits are improperly denied (false negatives) than we do about preventing these types of mistakes in the first place.

In one experiment, more than 900 participants recruited on Amazon Mechanical Turk were queried for their thoughts on mistakes related to punishments and rewards, and their sense of which mistakes it was more important to prevent from happening, and which mistakes it was more important to fix after the fact. 

Participants were presented three different scenarios that broadly fell under the rubric of punishments or rewards:

• Punishments:
  • Docking pay for poor performance
  • Charging higher insurance premiums for unsafe driving
  • Levying fines for using too much water during a drought
• Rewards:
  • Issuing bonuses for good work performance
  • Offering premium discounts for safe driving
  • Bestowing tax credits for conserving water during a drought

For punishments, there was more concern about making sure that those who deserved punishments got them upfront (preventing a false negative) than about chasing down people who evaded punishments after the fact (fixing a false negative). And it was deemed more important to rectify cases where a punishment was handed out to someone who didn’t deserve it (fixing a false positive) than to prevent this from happening in the first place (preventing a false positive). 

For rewards, participants said it was more important to prevent giving rewards to those who don’t deserve them (false positives) and better to fix cases where people who deserved rewards didn’t get them (false negatives).

Getting What You Deserve

Similar patterns were evident in the authors’ other experiments, as well. 

Rude and Shaddy state that “the overall effect is the endorsement of harsher policies in principle (before errors occur, when people focus more on preventing mistakes that help bad actors) than in practice (after errors occur, when people focus relatively more on fixing mistakes that harm good actors).”

Their data also exposes that participants had a stronger opinion about policies that seek to fix mistakes compared with policies designed to prevent mistakes. Rude and Shaddy note that while they have no position on whether such a preference is itself a mistake, it does suggest a potentially useful insight for policymakers. “Public support might be higher for policies that hew closer to the preferences observed in the fix frames.”