Lies, Damned Lies and Statistics — and Data Visualization

How we interpret time-series data is dependent on the designer’s chosen format

As the use of data visualization grows, so does the challenge to the public’s chart literacy. The Bharatiya Janata Party’s late 2018 tweetstorm of ridicule that followed the Indian government’s posting of an infographic featuring Prime Minister Narendra Modi is evidence we can sniff out obvious data-viz fails.

Truth of hike in petrol prices! pic.twitter.com/hES7murfIL

— BJP (@BJP4India) September 10, 2018

Not every misleading graphic has such glaring defects, though. Look at the bar chart below from the Daily Mail showing the UK’s GDP growth in the fourth quarter of 2016, and you’re likely to think there is a large difference in the data. There isn’t. It illustrates only a 0.1 percent adjustment from the Office of National Statistics’ first estimate.

On closer examination, one sees that the scale on the x-axis is designed to make the difference in the percentages look larger.

(More examples of questionable visualizations can be seen here and here.)

UCLA Anderson’s Stephen A. Spiller, University of Colorado’s Nicholas Reinholtz and University of Toronto’s Sam J. Maglio find, in research forthcoming in Management Science, that even in the absence of visual trickery or misleading infographics, the viewer’s judgment can still be swayed by the designer’s choice in how to present a chart’s data.

The paper explains that the exact same data can be assessed positively when shown in one format (such as the change in values over a period, as with Indian diesel prices) and negatively when formatted in another (such as the actual values per period). This matters, as data visualization is being used more and more in managerial decisions and in the media as shorthand for the vast amounts of information people and organizations process.

The researchers studied how people respond to time-series charts, one of the most ubiquitous visualizations in the media. The creator of a time-series visualization must make the decision to format the data as either a “stock,” which expresses the quantity at each point, or as a “flow,” the change in the quantity from one point to the next.

The table below illustrates data collected over six time periods shown in the stock and flow formats.

As the charts below show, the same data can look vastly different, depending on which format is chosen.

The researchers studied whether different presentation options with time-series charts influence interpretation. They randomly presented 100 participants with charts that use identical U.S. private-sector jobs data, similar to those below, from January 2007 to December 2013.

Participants were asked to evaluate the effect that President Obama had on the American economy during his first year in office. Opinions varied widely, depending on which graphic the participants viewed. Those viewing the stock chart (right), which showed a declining number of jobs continuing in Obama’s first term, evaluated his impact on the economy negatively; those viewing the flow chart (left), which showed an increasing trend reflecting a reduction in job losses, evaluated Obama’s impact on the economy positively. The difference in format led to a divergence in their qualitative judgments.

While not included in the researchers’ study, U.S. Gross Domestic Product, a widely accepted yardstick of economic performance, is another example of a statistic that is often misunderstood in the flow versus stock scenario. It’s usually reported as a flow, with the annualized rate of increase during a quarter compared to the rate of increase in the previous quarter, but it’s often written about in the media as if the actual amount, or stock, is being discussed. Articles discussing the “slowing economy” are often discussing an economy that is growing — just at a decelerating pace compared to the previous quarter, say, from 2.4 percent growth to 2.2 percent growth.

Corporations use a related phenomenon, cumulative versus periodic, to their advantage when presenting some results. In 2013, Tim Cook presented the explosive cumulative growth of Apple’s iPhone sales but Quartz responded with a doctored version of that chart showing iPhone sales falling over the most recent three quarters.

The stock versus flow presentations influence us on a more personal level when we’re viewing our investment returns. If data are presented in stock format, and an account is shown to be increasing, albeit at a slowing rate, investors might view it favorably. But the same investment results might be viewed negatively if they are shown to the investor in the flow format.

In their series of studies, the researchers also investigated whether forces beyond the presentation format caused the differences in interpretation. They checked different cross sections of the time series, differences in presentation details, potential misunderstanding of the data by the participants, and the participants’ political inclinations. Even as the researchers accounted for the effects of those variables, they found that the differences in interpretation by format persisted.

Trends in the data representation were also found to be important when participants were asked to forecast the future direction of the data. When people are presented with a time series in the flow format that ends in an upward trend, they tend to make optimistic forecasts — even when the net flow is still negative. A job market losing 200,000 jobs per month is encouraging if the prior month showed 300,000 lost jobs.

This suggests that a constant negative change would be evaluated as quite negative when represented as a stock condition because the trend is steeply downward sloping. But the evaluation is rather neutral in the flow condition because the trend is flat.

While the researchers are agnostic about the choice of presenting data as a stock or a flow, they see consistent evidence that the choice can impact the judgment of the viewer and can be used by the graphic’s creator to influence opinions. Viewer beware.