Overconfidence and subjective probabilities

Investing involves the application of imperfect information to subjective probability assessments in a complex, multivariable system. Successful investing requires the investor to be both different and right more often than not in his or her probability assessments. In an ideal world, we would train (“calibrate”) our forecasting skills by making a high number of subjective probability judgments in one subject area and receiving timely feedback after each event. Unfortunately, investors only make a limited number of investment decisions through their careers (compared to poker players, for example) and feedback can take months if not years to receive. In such a system, how can investors hope to improve their judgment, and by extension, their investment performance?

Clients of Montaka and followers of our blog will know that an integral aspect of our investment approach is that we think probabilistically rather than deterministically. This is to say that we view investment outcomes as probability distributions rather than point estimates. Ultimately this boils down to two things: i) our level of knowledge; and ii) our confidence in our level of knowledge. (A deterministic mentality would stop at (i)). Getting (i) right is important, but getting (ii) wrong can be disastrous. Believing something to be true when it just ain’t so will do far more damage to your returns when your confidence in that belief is high than when it is low.

Several years ago, Michael and Andrew Mauboussin created a probability calibration assessment to observe the relationship between knowledge and confidence (you can take the quiz yourself here). The purpose of the assessment was not to test a participant’s knowledge base (whether they were right or wrong), but rather their conviction in their knowledge base by assigning a subjective probability to how likely they thought their answer was correct (50% is a guess, 100% is a sure thing). The results of 11,000 participants were recently released, and some interesting insights can be drawn from the data, chief among which is that people tend to be overconfident on average.

Source: Andrew Mauboussin, Medium

The graph above plots the aggregate results of 550,000 responses. People tend to be 50% correct on their guesses, which is to be expected; however, as reported confidence increases, the average percentage correct fails to keep up. At 100% confidence (i.e. a sure thing), the average person was correct only 78% of the time – a clear indicator of overconfidence. This is an important takeaway for investors, because our positions tend to be sized according to our conviction levels, and if we are overconfident on average, our position sizes are likely to be too large relative to the uncertainties that exist. This is especially the case for a deterministic thinker, because to them, something either is or isn’t. Even having a bull/base/bear case is deterministic thinking if those cases are simply point estimates at different values. Thus, the first step towards probabilistic thinking is recognising that humans tend to be overconfident, and reality is much more like a probability distribution than we initially expect.

The two most common culprits of our overconfidence are confirmation bias and availability heuristic. Confirmation bias occurs when people jump to a conclusion and then search for (or recollect) information that confirms their initial conclusion. This is equivalent to buying a stock first, and then doing the research to support your purchase. Availability heuristic is a mental shortcut where we rely on immediately recallable examples when making a judgment. For example, one question in Mauboussin’s quiz asks whether Milan, Italy sits at a higher latitude than Toronto, Canada. Confirmation bias is when someone knows Toronto is higher latitude and thinks of the Rocky Mountains, ice hockey and freezing winters to confirm their answer. Availability heuristic is when someone doesn’t know the answer, but uses what they can immediately recall about Canada (snow-capped mountains) and Italy (canals and rolling Tuscan hills) to deduce that Toronto is higher latitude than Milan. In fact, Milan is about 2 degrees further north than Toronto.

Source: Andrew Mauboussin, Medium. The higher the confidence level, the more likely the answer was incorrect.

The second step towards probabilistic thinking is training your mind to translate uncertain events into probabilities. Research from leading behavioural psychologists such as Daniel Kahneman, Philip Tetlock and the late Amos Tversky indicate that people typically have a “three setting” mindset with respect to uncertain events: “a sure thing”, “impossible”, and “maybe”. Evolution programmed us this way – the intuitive yes/no/maybe judgment system was much better attuned to keeping our ancestors alive on the African savannah than was sitting down to deliberately calculate the probability that the shadow in the tall grass is a lion.

Source: Andrew Mauboussin, Medium

Consider the histogram above. This chart clearly captures the “three setting” mind in action – an overwhelming percentage of “maybes”, and a substantial percentage of “absolutely certains” (note that “sure thing” and “impossible” are both 100% confidence on a True/False quiz). It suggests that people struggle to assign a range of subjective probabilities to uncertain events, or in this case, the level of their own knowledge, either out of laziness or lack of practice. For an investor, “maybe” is usually not good enough, and is the symptom of a lazy research process that does not respect uncertainty.

Perhaps the more important takeaway from the histogram is that the number of responses falls away as the reported confidence moves from 60% to 90%. It appears that people are more comfortable assigning subjective probabilities slightly higher than maybe (60%-70%), but less inclined to make assessments of higher probabilities (80%-90%). This has significant implications, as it suggests that once people reach a certain probability threshold, they tend to jump straight to “a sure thing” or “impossible”, rather than making a distinction between a 90% probability event and a certain event. When it comes to investing, the difference between 90% and 100%, or 10% and 0%, can be the difference between appropriately managing a tail risk and blowing up your portfolio.

While the foregoing discussion does not offer any easy solutions to better investment performance, it is always helpful to be aware of behavioural biases that may be adversely affecting your investment process. Overconfidence is a leading cause of poor investment returns, so it’s best not to compound that pitfall by thinking about uncertain events with a deterministic mindset. No one wants to be the guy who is often wrong, but never in doubt.

DH5_2155Daniel Wu is a Research Analyst with Montaka Global Investments. To learn more about Montaka, please call +612 7202 0100.

Leave a Comment

Your email address will not be published. Required fields are marked *