Research Brief

Lies, Damned Lies and Statistics — and Data Visualization

How we interpret time-series data is dependent on the designer’s chosen format

As the use of data visualization grows, so does the challenge to the public’s chart literacy. The Bharatiya Janata Party’s late 2018 tweetstorm of ridicule that followed the Indian government’s posting of an infographic featuring Prime Minister Narendra Modi is evidence we can sniff out obvious data-viz fails.

Not every misleading graphic has such glaring defects, though. Look at the bar chart below from the Daily Mail showing the UK’s GDP growth in the fourth quarter of 2016, and you’re likely to think there is a large difference in the data. There isn’t. It illustrates only a 0.1 percent adjustment from the Office of National Statistics’ first estimate.

Opt In to the Review Monthly Email Update.

On closer examination, one sees that the scale on the x-axis is designed to make the difference in the percentages look larger.

2016 Q4 Growth Upgraded. Image shows a 0.60% in the first estimate and a 0.70% in the second estimate.

(More examples of questionable visualizations can be seen here and here.)

UCLA Anderson’s Stephen A. Spiller, University of Colorado’s Nicholas Reinholtz and University of Toronto’s Sam J. Maglio find, in research forthcoming in Management Science, that even in the absence of visual trickery or misleading infographics, the viewer’s judgment can still be swayed by the designer’s choice in how to present a chart’s data.

The paper explains that the exact same data can be assessed positively when shown in one format (such as the change in values over a period, as with Indian diesel prices) and negatively when formatted in another (such as the actual values per period). This matters, as data visualization is being used more and more in managerial decisions and in the media as shorthand for the vast amounts of information people and organizations process.

The researchers studied how people respond to time-series charts, one of the most ubiquitous visualizations in the media. The creator of a time-series visualization must make the decision to format the data as either a “stock,” which expresses the quantity at each point, or as a “flow,” the change in the quantity from one point to the next.

The table below illustrates data collected over six time periods shown in the stock and flow formats.

TimeStockFlows
138
215-23
3249
44218
5508
66010

As the charts below show, the same data can look vastly different, depending on which format is chosen.

Image displays two graphics side by side. The first shows a strong upward trend and while the second shows a less stable, more varying trend.

The researchers studied whether different presentation options with time-series charts influence interpretation. They randomly presented 100 participants with charts that use identical U.S. private-sector jobs data, similar to those below, from January 2007 to December 2013.

Image shows two graphics side by side. Both graphs display the same data points but do so in reverse ways which creates the illusion of conflicting images.

Participants were asked to evaluate the effect that President Obama had on the American economy during his first year in office. Opinions varied widely, depending on which graphic the participants viewed. Those viewing the stock chart (right), which showed a declining number of jobs continuing in Obama’s first term, evaluated his impact on the economy negatively; those viewing the flow chart (left), which showed an increasing trend reflecting a reduction in job losses, evaluated Obama’s impact on the economy positively. The difference in format led to a divergence in their qualitative judgments.

While not included in the researchers’ study, U.S. Gross Domestic Product, a widely accepted yardstick of economic performance, is another example of a statistic that is often misunderstood in the flow versus stock scenario. It’s usually reported as a flow, with the annualized rate of increase during a quarter compared to the rate of increase in the previous quarter, but it’s often written about in the media as if the actual amount, or stock, is being discussed. Articles discussing the “slowing economy” are often discussing an economy that is growing — just at a decelerating pace compared to the previous quarter, say, from 2.4 percent growth to 2.2 percent growth.

Corporations use a related phenomenon, cumulative versus periodic, to their advantage when presenting some results. In 2013, Tim Cook presented the explosive cumulative growth of Apple’s iPhone sales but Quartz responded with a doctored version of that chart showing iPhone sales falling over the most recent three quarters.

Apple's iPhone sales chart but Quartz responded with a doctored version of that chart showing iPhone sales falling over the most recent three quartersAs published by The VergeQuartz enhancement

The stock versus flow presentations influence us on a more personal level when we’re viewing our investment returns. If data are presented in stock format, and an account is shown to be increasing, albeit at a slowing rate, investors might view it favorably. But the same investment results might be viewed negatively if they are shown to the investor in the flow format.

In their series of studies, the researchers also investigated whether forces beyond the presentation format caused the differences in interpretation. They checked different cross sections of the time series, differences in presentation details, potential misunderstanding of the data by the participants, and the participants’ political inclinations. Even as the researchers accounted for the effects of those variables, they found that the differences in interpretation by format persisted.

Trends in the data representation were also found to be important when participants were asked to forecast the future direction of the data. When people are presented with a time series in the flow format that ends in an upward trend, they tend to make optimistic forecasts — even when the net flow is still negative. A job market losing 200,000 jobs per month is encouraging if the prior month showed 300,000 lost jobs.

This suggests that a constant negative change would be evaluated as quite negative when represented as a stock condition because the trend is steeply downward sloping. But the evaluation is rather neutral in the flow condition because the trend is flat.

TimeStockFlows
10
-2-5-5
3-10-5
4-15-5
5-20-5
6-25-5
Image displays two graphs side by side. The left side shows an increasing downward slope. The right shows a stable horizontal line.

While the researchers are agnostic about the choice of presenting data as a stock or a flow, they see consistent evidence that the choice can impact the judgment of the viewer and can be used by the graphic’s creator to influence opinions. Viewer beware.

Featured Faculty

  • Stephen Spiller

    Associate Professor of Marketing and Behavioral Decision Making

About the Research

Spiller, S., Reinholtz, N., & Maglio, S. (in press). Judgments based on stocks and flows: Different presentations of the same data can lead to opposing inferencesManagement Science.

Related Articles

Research Brief / Data Analytics

Old Ways of Thinking Cause Managers to Disregard Analytics

Highlighting the right data and making it relevant can help overcome resistance

A wooden man with a red tie leans out of a line of nondescript wooden men Research Brief / Competition

Gaining Status in an Online Community

How unknown individuals turn into influencers on a platform for programmers

A white clock on a black background that is dissolving on the right side Research Brief / Data Analytics

Deciding When to Decide Can Lead to Better Outcomes

An interpretable model versus black-box algorithms for complex decision making

Illustration of two men with one crossing his fingers behind his back Research Brief / Behavioral Decision Making

Being Biased against Friends to Appear Unbiased

If the boss is your friend, and compensation decisions are public, a bonus you’d get on merit might not be forthcoming