Old Ways of Thinking Cause Managers to Disregard Analytics

Highlighting the right data and making it relevant can help overcome resistance

Michael Lewis’s book, Moneyball, showed how hard it is for decision-makers — such as major-league scouts for the Oakland Athletics — to set aside their old habits in favor of data-derived analytics. The problem isn’t restricted to baseball.

Algorithm-based decision support systems, or DSS, are designed to help users choose the best course of action, whether it’s the most efficient route for delivery drivers or the optimal time for farmers to plant crops. Used properly, a DSS can make decision-making less subjective and biased and, in theory, make operations more efficient and profitable.

Trouble is, managers often disregard the DSS’ recommended course of action because — like the Oakland coaches — they trust their own intuition and experience more. If that happens often enough, business performance can suffer.

A working paper from UCLA Anderson’s Felipe Caro and IESE Business School’s Anna Sáez de Tejada Cuenca examines why managers deviate from a DSS’ recommendations. Using data from Zara, the Spanish fast-fashion retailer, they show how managers setting clearance-sale prices demonstrated three common behavioral biases when rejecting the recommendations of their DSS:

• Status quo bias: Managers were reluctant to embrace the recommendations of the new system, preferring to stick with the rules of thumb they used previously.
• Salience bias: Before the DSS, they marked down prices based on how quickly inventory levels were falling, with the goal of having zero inventory at the end of the sales period. Inventory levels were more visible, or salient, than less tangible metrics, such as revenues.
• Rational inattention: Managers, who had to set prices on hundreds of sales items every week, would reduce the number of products they had to monitor by combining sales categories more quickly than the DSS recommended, even if it left money on the table.

As a result, Zara managers often ignored the DSS’ recommendations, in some cases more than half the time, and the system didn’t deliver the increases in revenues promised during an early pilot.

“Analytics-based tools can help achieve the best possible outcomes, but only if managers believe in them,” the authors write.

In 2008, Zara adopted a DSS to help managers set prices during semiannual clearance sales. Based on demand forecasts, the system would recommend weekly markdowns with the goal of maximizing revenues; managers still had final say. A pilot study of the system found that its recommendations increased clearance-sales revenues by nearly 6%.

After the DSS was rolled out companywide, though, managers followed its recommendations far less than expected. During the first clearance sale after the rollout in 2010, managers in countries where Zara owned the stores accepted the DSS’ pricing recommendation 61% of the time and about 46% of the time in countries with franchisee-owned stores. Managers frequently lowered prices when the DSS proposed keeping them unchanged.

Before the DSS, managers made their markdown decisions based on weekly reports showing inventory levels and a projection of run-out times, when products would sell out, with the goal of moving 90% of the initial stock before the end of the sales period. When the DSS arrived, its user interface was developed mimicking the legacy reports, emphasizing inventory levels and run-out times.

Managers’ unwillingness to follow the DSS’ recommendations, even when it resulted in lower-than optimal revenues, was evidence of their status quo bias, the authors observe. They were reluctant to abandon their usual decision-making shortcuts, such as run-out times and fewer price categories. 

Other biases also played a role. The DSS often suggested prices that would leave unsold inventory when the sale ended. That’s not a big deal for revenue-management experts, who understand that occasionally it’s better to leave some inventory unsold if overall revenues are higher, the authors write.

But such a recommendation is counterintuitive for managers with a single-minded focus on inventories. So managers in the study were likely to disregard the DSS when it appeared a category might end up with large leftover inventories. This, the authors say, was consistent with inventories being more salient to managers than revenues.

Rational inattention bias came into play as managers had to set prices for more than 200 categories and to watch multiple metrics for each category. They would frequently combine product categories despite the DSS’ recommendations just to reduce the amount of data they had to track. The more categories managers worked with, the more likely they were to deviate from the DSS’ recommendations.

Still, the study suggests, managers can mitigate their biases, especially the saliency bias of inventory, which had the most prominent effect.

At Zara, the most important measure of a sale’s success is the ratio between sale revenue and the value of regularly priced inventory, called Y, and the objective of the DSS is to maximize that ratio. Zara added a field for Y to the DSS before the spring 2011 sale, to little effect. A year later, Zara revised the dashboard again, this time to display the previous year’s Y ratio so to give managers a benchmark of their progress.

The results were better, especially for managers in franchise countries. After both interventions, the share of those managers who followed the DSS recommendations increased 9.6 percentage points, or 21%, from before the changes. Managers for countries where Zara owned the stores — who had already begun to take Y into account even before it was added to the DSS — increased their adherence slightly.

Together, the findings reinforce the importance of a well-designed DSS interface. It isn’t enough to present the old metrics in a new way; the DSS needs to emphasize the desired goal: increased revenues. “The path to higher adherence did not consist in structural changes to the underlying optimization,” the authors write, “but in small changes on the information presented to managers.”