The experts have been having a bad run of late. Most publications and experts were red in the face after the US elections results were declared. Brexit took the experts by surprise. Closer home, the experts on popular television have held forth with utmost confidence on everything from poll outcomes to predicting winners in sports. None of these experts are back on air to explain why they were so wrong. When they are back on air predicting outcomes of the next election or match, the viewers too forget their track record of success.
IBM's Thomas Watson said that there was room for only five computers in the world. That was 1943. Atari rejected Jobs' offer to build a personal computer. Microsoft missed the music, mobile and social waves. Google Wave, Google+ are pinned against the search giant’s name as missed opportunities. Is it only experts who are guilty of getting their predictions wrong?
Philip Tetlock conducted a set of small-scale forecasting tournaments between 1984 and 2003. The forecasters were 284 experts from a variety of fields, including government officials, professors, journalists, and others, with many opinions, from Marxists to free-marketeers.
The tournaments solicited roughly 28,000 predictions about the future and found the forecasters were often only slightly more accurate than a coin toss. Forecasters with the biggest news media profiles were also especially bad. Translated, that means that the more these experts were quoted, the more likely they were to be wrong.
Organisations hire the wrong CEOs. The CEOs hire the wrong team members. Students choose the wrong subjects to study and then fret whether they should cut their losses and start afresh or just hold their breath till it gets over. People make wrong choices when they choose their partners. So if everyone makes mistakes, why do we expect experts to get it right?
Why do experts go wrong? Being an expert means having in-depth knowledge of a subject. One way to understand this scenario is to think of an expert’s view as being on what you read on Facebook. On Facebook most people connect with those who echo their world view. We do not 'friend' someone whose opinions contradict our own. If someone shares a view that does not reflect the opinion of the majority, they would hesitate to share it or may tone it down. In that dilution the essence is lost. Depth of knowledge often comes at the cost of insights that could come from someone who is applying a different lens.
So should you trust generalists instead? While the specialists know a hundred things about one subject, the generalists know one thing about a hundred things. If they cannot filter out the signal from the noise they will get overwhelmed. That is a skill that takes time to build and sharpen. New research from MIT suggests that employees with a diverse Twitter network — one that exposes them to people and ideas they don’t already know — tend to generate better ideas.
Working in transdisciplinary teams is a good way to overcome the specialist-versus-generalist debate. Being able to balance hard core data crunching skills with human insights may be the sweet spot we are looking for.
(Disclaimer: The views and opinions expressed in this article are those of the author and do not necessarily reflect the views of YourStory.)