The standard approach for teasing out this consequential behavioral bias doesn’t account fully for ability
If there were a pantheon of behavioral biases that can cloud optimal decision making, overconfidence would likely reign supreme. Nobel Laureate Daniel Kahneman, the OG of behavioral economics, designated it “the most significant of the cognitive biases.”
While there can be upside to being overly confident — it is hard to imagine the entrepreneurial ecosystem that drives innovation without it — research Kahneman and his colleague Amos Tversky published 50 years ago is the foundational basis for a trove of subsequent work that has exposed the negative consequence when perception of our ability exceeds our actual ability.
Overconfident investors (typically men) trade more and earn lower returns than more patient investors. Overconfident CEOs underestimate business risks and overestimate potential payoffs, leading to poor capital allocation decisions. Older people prone to overconfidence in their memory skills are more likely to make errors, such as forgetting to take medication as prescribed. Overconfident individuals are often perceived as being smarter and more capable than they actually are, which can devolve into leadership based on personality and not talent.
Yet in a provocative paper forthcoming in Management Science, UCLA Anderson’s Stephen A. Spiller suggests there is a substantive flaw underpinning the study of overconfidence. Deconstructing the common methodologies used to capture overconfidence, he makes a case that the role of a participant’s ability is not fully captured or isolated.
“When research indicates that differences in overconfidence are associated with other behaviors, beliefs or evaluations, researchers should account for the possibility that differences in ability could provide a sufficient explanation,” Spiller writes.
The Problem with Measuring Overconfidence
A typical way to measure overconfidence is to have study participants take a quiz of some sort and simultaneously ask them how well they think they did. Answer 7 of 10 trivia questions correctly while self-reporting that you think you nailed 9 of 10, can be a sign of overconfidence. There are a handful of ways to statistically parse the data, and Spiller shows how three approaches all have the same Achilles’ heel: They fail to fully separate confidence from ability.
He suggests that a quiz is a “noisy proxy” for measuring ability, given the potential for a lucky guess, or the messiness of a poorly constructed question, or embedded errors in how the information is analyzed.
Moreover, he asserts that the self-reporting side of the equation has an embedded tie to ability that is overlooked; our self-reporting on how we performed on some task is likely affected by our knowledge about how good (or not) we are on this topic/skill. Yet the methodology doesn’t factor in how one’s perceived ability may be impacting the self-reporting. What is seen as overconfidence may to some extent be a reflection of personal intel of ability.
Revisiting Overconfidence Research
In 2008, “The Trouble with Overconfidence,” published in Psychological Review, proposed that overconfidence comes in three distinct flavors: We overestimate our ability, we over-rate our ability relative to others, and we are oversure that we are right. Participants completed a series of 18 trivia quizzes. The researchers found that the difficulty of a given quiz impacts what type of overconfidence was most at play.
Spiller dove back into the original dataset, leveraging the fact that each participant took a series of quizzes. Being able to analyze participant data from test to test enabled him to tease out the potential role of ability.
If overconfidence is purely a bias, there should be no predictive power of overconfidence from one quiz to performance on the next quiz. That is, if you merely think you’re smarter than you really are, you won’t have better performance on a subsequent quiz. But in two of the three measures of overconfidence he found that participants who reported higher confidence in one quiz had significantly better performance in a subsequent quiz. That is, participants who had a higher perception of their ability indeed did better on subsequent quizzes, suggesting that ability, not just overconfidence, is in the mix.
Spiller revisited another piece of research that claimed to show how overconfidence causes observers to view that person as more competent. Spiller deconstructed the data and asserts that the higher opinion of that person can be explained by their actual ability.
He also reviewed a third piece of prior research that places overconfidence in the realm of social inequality: People with higher social class were shown to have greater overconfidence. But in reanalyzing the data, Spiller asserts that people in higher social classes weren’t necessarily overconfident, but rather, they were better at the actual quizzes (ability) which could be a function of the resources available to higher social classes.
Spiller acknowledges eliminating the confound of ability in overconfidence research “may be a nearly unattainable target,” but he proposes a series of research techniques that in concert would likely reduce its impact. Moreover, he recommends that researchers run their findings through the same prism he explored: To what extent, if any, is what is ascribed to be overconfidence also a function of ability.
Featured Faculty
-
Stephen Spiller
Professor of Marketing and Behavioral Decision Making
About the Research
Spiller, S.A. (2026). (in press). Widely-Used Measures of Overconfidence Are Confounded with Ability. Management Science.