Experts are bad at forecasting: Remember this, next time you see a forecast

Oct 7, 2015

- By Vivek Kaul

Vivek Kaul
If I ever have the time, the money and the resources, I would like to carry out an experiment. Every day on business TV channels, experts offer their forecasts on stock prices, commodity prices, the direction of the economy, politics of the nation and so on.

There are other experts making forecasts through research reports. As the British economist John Kay writes in Other People's Money: "Most of what is called 'research' in financial sector would not be recognised as research by anyone who has completed an undergraduate thesis."

Getting back to the topic at hand, I would like to figure out how many of these forecasts eventually turned out to be correct. So, if an analyst says that he expects the price of HDFC Bank to cross Rs 1,300 per share in a year's time, did he eventually get it right.

Also, I would like to figure out whether the "so-called" forecasts were forecasts at all, in the first place? Saying that the HDFC Bank stock price will cross Rs 1,300 per share, but not saying when, is not a forecast. As Philip Tetlock and Dan Gardner write in their new book Superforecasting-The Art and Science of Prediction: "Obviously, a forecast without a time frame is absurd. And yet, forecasters routinely make them."

When it comes to the stock market, there are two kinds of experts who come under this category of making a forecast without a time-frame attached to it. One category is of those who keep saying that the bull market will continue, without really telling us, until when. "Predicting the continuation of a long bull market in stocks can prove profitable for many years-until it suddenly proves to be your undoing," write Tetlock and Gardner.

Advertisement
  Pick Stocks Like a Pro  
  Do you want to pick stocks like Warren Buffett, the master investor?

Or for that matter, the other investing greats?

We know you, like any smart investor, are always looking for opportunities to learn from these brilliant stock pickers.

And that's why, for the first time ever at Equitymaster, we have revealed what we believe are the key secrets behind success of the masters...

You can discover seven of these secrets in our latest special report - The 7 Secrets of a Shrewd Value Investor.

This report is FREE.

We recommend you claim your copy right away... and set yourself on the path to a successful stock picking future.

For full details on how you can claim this report, just click here...
 

The second category is of those who keep saying that the bear market is on its way, without saying when. "Anyone can easily "predict" the next stock market crash by incessantly warning that the stock market is about to crash," write Tetlock and Gardner.

The broader point is that no one goes back to check whether the forecast eventually turned out to be correct. There is no measurement of how good or bad a particular expert is at making forecasts. I mean, if an expert is constantly getting his forecasts wrong, should you be listening to him in the first place.

But no one is keeping track of this, not even the TV channel.

As Tetlock and Gardner write: "Accuracy is seldom even mentioned. Old forecasts are like old news-soon forgotten-and pundits are almost never asked to reconcile what they said with what actually happened." And since no one is keeping a record, it allows experts to keep peddling their stories over and over again, without the viewers knowing how good or bad their previous forecasts were.

"The one undeniable talent that talking heads have is their skill at telling a compelling story with conviction, and that is enough. Many have become wealthy peddling forecasting of untested value to corporates executives, government officials, and ordinary people who never think of swallowing medicine of unknown efficacy and safety," write Tetlock and Gardner.

In fact, in the recent past, many stock market experts were recommending midcap stocks. After the Sensex started crashing the same set of experts asked investors to stay away from midcap stocks as far as possible.

There is a great story I was told about an expert, who was the head of the commodities desk at one of the big brokerages. He was also a regular on one of the television channels as well. This gentlemen kept telling the viewers to keep shorting oil for as long as prices were going up and then when the prices started to fall, he asked them to start buying. This was exactly opposite of what he should have been recommending. Obviously anyone who followed this forecast would have lost a lot of money.

I can say from personal experience that predicting the price of oil is very difficult, given that there are so many factors that are at work. As Tetlock and Gardner write: "Take the price of oil, long a graveyard topic for forecasting reputations. The number of factors that can drive the price up or down is huge-from frackers in the United States to jihadists in Libya to battery designers in Silicon Valley-and the number of factors that can influence those factors is even bigger."

Nevertheless, the television appearances of the commodity expert I talked about a little earlier, continue. And why is that the case? Tetlock and Gardner provide the answer: "Accuracy is seldom determined after the fact and is almost never done with sufficient regularity and rigor that conclusions can be drawn. The reason? Mostly it's a demand-side problem: The consumers of forecasting-governments, businesses, and the public don't demand evidence of accuracy. So there is no measurement. Which means no revision. And without revision, there can be no improvement." And so the story continues.

One would like to believe that forecasts are made so that people can look into the future with greater clarity. But that is not always the case. Some forecasts are made for fun. Some other forecasts are made to fulfil the human need to know what is coming. Some other forecasts are made to advance political agendas.

And still some other forecasts are made to comfort people "by assuring [them] that their beliefs are correct and the future will unfold as expected," Tetlock and Gardner, point out. Now only if it were as simple as that.

In fact, Tetlock spent close to two decades following experts and their forecasts. In the experiment, Tetlock chose 284 people, who made a living by predicting political and economic trends. Over the next 20 years, he asked them to make nearly 100 predictions each, on a variety of likely future events. Would apartheid end in South Africa? Would Michael Gorbachev, the leader of USSR, be ousted in a coup? Would the US go to war in the Persian Gulf? Would the dotcom bubble burst?

By the end of the study in 2003, Tetlock had 82,361 forecasts. What he found was that there was very little agreement among these experts. It didn't matter which field they were in or what their academic discipline was; they were all bad at forecasting. Interestingly, these experts did slightly better at predicting the future when they were operating outside the area of their so-called expertise.

It is well-worth remembering these lessons the next time you come across a forecast. And that includes the forecasts made in The Daily Reckoning as well.

Publisher's Note: Vivek Kaul, the India Editor of the Daily Reckoning, just made a bold call - Real Estate prices are headed for a fall. Well, if you are someone who is looking to buy real estate, or is just interested in the space, I recommend you read Vivek's detailed views in his just published report "The (In)Complete Guide To Real Estate". To claim your copy of this Free Report, just reconfirm your Free subscription to the Daily Reckoning...

Vivek Kaul is the Editor of the Diary and The Vivek Kaul Letter. Vivek is a writer who has worked at senior positions with the Daily News and Analysis (DNA) and The Economic Times, in the past. He is the author of the Easy Money trilogy. The latest book in the trilogy Easy Money: The Greatest Ponzi Scheme Ever and How It Is Set to Destroy the Global Financial System was published in March 2015. The books were bestsellers on Amazon. His writing has also appeared in The Times of India, The Hindu, The Hindu Business Line, Business World, Business Today, India Today, Business Standard, Forbes India, Deccan Chronicle, The Asian Age, Mutual Fund Insight, Wealth Insight, Swarajya, Bangalore Mirror among others.

Disclaimer: The views mentioned above are of the author only. Data and charts, if used, in the article have been sourced from available information and have not been authenticated by any statutory authority. The author and Equitymaster do not claim it to be accurate nor accept any responsibility for the same. The views constitute only the opinions and do not constitute any guidelines or recommendation on any course of action to be followed by the reader. Please read the detailed Terms of Use of the web site.

Recent Articles

Using Deposits to Rescue Banks is a Bad Idea; It Needs to Be Nipped in the Bud December 11, 2017
The Financial Resolution and Deposit Insurance(FRDI) Bill proposes to do just this.
5 Agents of Change Investors Need to Know About Now December 11, 2017
The world is changing fast right now in ways that many investors might not easily recognize or want to admit. If you're not paying attention, you could be letting opportunities pass you by without even realizing it.
Why Economists Are Morons December 9, 2017
Why are most modern economists such nincompoops? The quick answer: because it pays.
Bitcoin is the Best Offshore Bank Account December 8, 2017
The biggest advantage bitcoin has over offshore banks is that it's truly offshore. Not just off Britain's shores, but off every country's shores.

Equitymaster requests your view! Post a comment on "Experts are bad at forecasting: Remember this, next time you see a forecast". Click here!

5 Responses to "Experts are bad at forecasting: Remember this, next time you see a forecast"

Praveen

Oct 11, 2015

Hi,

Good article. Completely agree.

"Vivek Kaul, the India Editor of the Daily Reckoning, just made a bold call - Real Estate prices are headed for a fall."

Did Mr. Kaul write this article "Experts are bad at forecasting..." before making the "bold call"?

I am not suggesting that his call will go wrong. I have not even read it. All I am saying is forecasting is no good.

Like (1)

Satish M

Oct 8, 2015

I have observed another common comment by these TV pundits that is to say that "Nifty is stuck in a range" and then proceed to give the range.One must realize that a range can be specified for any stock or any index at all times when the time frame is not included, without being wrong.So if one says Nifty has a range of 5000 points,the statement will cover a decade probably or if a range of 1000 points is mentioned,it can cover several months easily.So this statement is absurd and useless.So what these "experts" do is to keep changing the range values on a daily basis to give an appearance of having an insight into the market behaviour.

Like (1)

SundaraVaradan

Oct 7, 2015

Unverifiable Forecasts:
Thank you. Many of us are feeling the same (frustration). Media is happy to have their media TIME/Space filled-up!

What we need is ANALYST RATING. This must be displayed, whenever the Analyst forecasts!
Hope Indian Media + Viewers insist on this Credit Rating!

Like (1)

abhay Dixit

Oct 7, 2015

suggest read Thanking -- Fast and slowpage 245-6-7 about forecasting curriculum preparation/book writing by experts.

Like (1)

Prashanth

Oct 7, 2015

Great article.
As given in Little book of Behavioural Investing from James Montier, the stats shows
The analysts are no better. Their forecasting record is simply dreadful on both short- and long-term issues. When an analyst first makes a forecast for a company ’s earnings two years prior to the actual event, they are on average wrong by a staggering 94 percent. Even at a 12 -month time horizon, they are wrong by around 45 percent! To put it mildly, analysts don’t have a clue about future earnings. Their performance in divining the longer-term future is no better than their nonexistent ability to forecast the short term. When looking at the five-year growth rate forecasts from analysts versus the actual outcomes, a rather depressing reality asserts itself. The stocks that analysts expect to grow the fastest actually grow no faster than the stocks they expect to grow the slowest! Of course, buying the stocks that they expect to grow the fastest just means signing up for massive disappointment. For now we can content ourselves by saying that analysts have absolutely no idea about forecasting long - term growth.





Montier, James (2010-03-30). The Little Book of Behavioral Investing: How not to be your own worst enemy (Little Books, Big Profits (UK)) (p. 59). Wiley. Kindle Edition.

Like (1)
  
Equitymaster requests your view! Post a comment on "Experts are bad at forecasting: Remember this, next time you see a forecast". Click here!