Failing well: how to assess, move on and succeed after failures

Failure is part of the journey in entrepreneurship, so it helps to understand the different types of failure and their causes, and how to learn from examples of failed and successful businesses.

This is a rather long and rambling book, but shines the light on different types of failure. Megan McArdle, author of The Up Side of Down: Why Failing Well Is the Key to Success, is a columnist for The Washington Post and was earlier correspondent for The Atlantic and The Economist.

The 10 chapters in the 300-page book cover corporate scandals, movie flops, industry failures, financial crises, hospital errors, experts’ mistakes, education flaws, and even penal law. The material draws on studies by economists, psychologists and anthropologists.

Here are my eight takeaways from the book, in terms of types of failures and how to deal with them. See also my reviews of the related books The Wisdom of Failure, Fail Better, Fail Fast, and Failing to Succeed.

A fast-moving, interconnected and complex world calls for increasing skills in experimentation, iteration, self-reflection, and decision-making, says Megan. Frequent mistakes will alternate with occasional moments of triumph. Effective people fail early, often, cheaply, gracefully, and even heroically.

Failure and hardships are part of the journey to success, just as blisters are signs that a tennis player is practising hard. Failure and difficulty appear in different ways in various professions, such as writer’s block for authors and journalists.

1. Bouncing back from failure

Many corporate giants have experienced failure, sometimes in spectacular ways – but some have bounced back successfully, such as Apple from its “pre-success failure” Newton.

Recovering from setbacks also sets entrepreneurs onto new directions where they eventually find success. Colonel Sanders lost his truck stop due to construction of a new highway; he then explored the fried chicken business, going on famously to found KFC.

Losing a job can be depressing, but also leads you to explore new things and catalyse the move to a different and better life. Continuous networking, side activities, and volunteering in projects can open up new avenues.

It also helps to learn from those who face failure frequently in their daily jobs, eg. sales teams, who routinely face rounds of rejection before striking a good deal. Fear of rejection can be overcome by being clear on goals, documenting activities, preparing scripts and scenarios, and connecting with communities of practice.

Megan cites the popular aphorism in this regard: “Good judgement comes from experience, and experience comes from bad judgement.” Recovering from accidents or failures can also be memorable as a badge of resilience. This applies to personal habits and addictions as well as inter-personal relationships.

1. Obstacles to spotting and recovering from failure

Planning and testing are regarded as ways to avoid failure down the road, but even such experiments can have limits. Coca-Cola’s early tests showed that New Coke could work – but it lasted only three months in the market, and the company switched back to Classic Coke.

This is a good example of the “Curse of the Successful Pilot Project,” says Megan. Consumers may have said they liked the free samples of New Coke, which were distributed in parking lots – but that is very different from actually spending hard-earned money to buy it in a store.

Another cause of failure is “inattentional blindness” – the inability to see glaring errors because we are focused on something else. A notable example here is the failure of CBS news team Dan Rather and Mary Mapes to realise that one of their stories was built on documents that turned out to be fake.

GM succeeded early on in the auto industry with diverse models (“for every purse and purpose”) and financing options; but in the face of stiff competition and the economic crisis of 2009, it had to eventually cut back on employee benefits and dealer partnerships. Once-successful companies can have a hard time changing course, and can end up becoming victims of their past success.

“Pretending everything is fine even when it’s not is so common that cognitive scientists have given it a name: the normalcy bias,” says Megan. This leads to failure to change enough or respond fast enough.

Managers tend to escalate commitment to failed projects and end up clinging to their mistakes. It is human to suffer from loss aversion, or inability to admit mistakes, for reasons such as embarrassment or ego. “The problem is that we have a bad tendency to invest ourselves emotionally in our mistakes and cling to them until it is too late,” cautions Megan.

Lack of willingness to admit failure applies to individuals as well as groups, a phenomenon Megan calls “groupidity” or collective stupidity. Unfortunately, it is also human to have herd mentality.

A related phenomenon is “blamestorming,” or finding a scapegoat rather than admitting that the causes of failure are sometimes too complex. For example, the 2009 financial crisis led to lots of accusations against “Mad Hatters, Moochers, Con Men and Corporate Shills,” Megan explains. It is human to expect the logic and emotional comfort of an explanation.

Other reactions are conspiracy theories such as blaming the 9/11 terror attacks as a nefarious US government plot.

Sometimes, people also “bend the maps” or try to twist facts to suit their case and justify their mistakes. This is a result of confirmation bias or “recruiting evidence” to prop up a faulty proposition. Confirmation bias can prevent us from asking the right but tough questions; we also tend to dismiss criticism from those with different political leanings. It takes courage to admit mistakes; errors in judgement should not lead to a lapse in ethics, Megan cautions.

Consumer attitudes are tough to crack and change as well, leading to many product failures. Many people say they would like to eat healthy food, exercise well, and take medication regularly – but reality is often otherwise, and this can trip up many an ambitious project.

1. The role of culture and politics

“Markets turn out to rely as much on morality as monetary policy; sociability can be as important as structure,” observes Megan. Individual character along with socio-economic environment can be predictors of failure and ethics.

The US and Europe differ in the rules and culture surrounding risk and failure. Failure is a badge of honour and even the sign of a visionary risk-taker in Silicon Valley, but can be seen as a sign of irresponsibility and lead to the end of a career in Europe, says Megan. Bankruptcy rules in the US let founders began afresh with a clean slate, and are thus favourable to entrepreneurs.

“America is far and away the bankruptcy capital of the world. Everywhere else, bankruptcy is a stigma, an enduring disgrace, a permanent stain,” Megan explains. “By lowering the cost of failure, our bankruptcy laws make it easier for people to take risks in the first place, By freeing people from the sunk costs of their failed business, we free them up to try again,” she adds.

Human evolution from foraging to farming cultures also led to new types of risk appetite and moral attitudes in society. Foragers face greater uncertainty, have more mobility, and are more dependent on luck; farmers have more stability, less mobility, and are more dependent on hard work.

Thus, Megan argues that Europe treats entrepreneurial risk-taking like farming (a result of good planning and hard work), whereas the US treats it like foraging, with uncertain results and a big role for luck.

1. Failure and policy

Attitudes towards failure and fair treatment also differ in the US between conservatives and liberals; liberals worry about the fairness of equality while conservatives worry about the fairness of proportion. This in turn affects attitudes and policies towards taxation, safety nets, and welfare benefits for the jobless, such as unemployment insurance.

Social experiments can be harder to conduct and take longer to analyse, cautions Megan, drawing on decades of policy changes in the US. The same data can also lead to multiple interpretations depending on political bias.

1. Movies: hits and flops

Hollywood shows how hard it is to predict hits and flops even with research, expertise and experience on your side. Mega-production Titanic was expected to be a flop but turned out to be a huge hit – exactly the opposite of Waterworld.

Steven Spielberg had a string of hits – but also a flop (the period comedy 1941). “Past performance is no guarantee of future results,” cautions Megan. At the same time, human behaviour is not totally random; more experimentation is required to improve accuracy in complex systems, but experiments also have their limits.

1. Medical errors

The medical world teaches us to distinguish between accidents, mistakes and failures, Megan explains. Accidents could not have been expected or planned for; mistakes could have been anticipated but are not too serious; failures are more serious, and are the result of misdiagnosis or surgical carelessness.

System design, medical procedures, mechanical devices and human judgement are areas where things can go wrong. Other variances can happen due to less staffing on weekends, or patients’ lack of familiarity with new hospitals. Normative errors occur when procedures or norms are not properly adhered to by medical staff or doctors and shortcuts are adopted instead.

1. Market failures

Problems can arise in the way market governance rules are set up; unscrupulous players can find loopholes and game the system, as seen in energy crises in California, or in the fraudulent activities of companies like Enron.

“Markets work best when formal rules are supported by a strong set of informal moral rules,” says Megan. Techniques like experimental economics can help model such markets and detect failure patterns in advance, well beyond the capabilities of “blackboard economics.”

1. Long-term responses

Much of education focuses on the successes in science and literature, but not as much on the struggles before success was found. Success comes from hard work as much as flashes of brilliance, says Megan. Students should be given chances to fail as well as chances to recover; poorer schools unfortunately do not offer students good chances for recovery.

One of the concluding chapters in the book focuses on failures in the penal system to correct failures of character in criminals. Randomised draconian punishments or seeking revenge are not the answer; consistent enforcement of appropriate corrective measures is better. Megan even compares this to the “tough love” of parenting, with a combination of reward and punishment.

In sum, the book points the way ahead for us to make a systematic classification of errors, mistakes, failures, accidents, and disasters. The classification should also include predictability, impacts, recovery measures, and implications for future planning. Cultural attitudes towards failure in other parts of the world can be mapped out as well.

The experience of failures can be charted across different organisational sizes and sectors, with appropriate documentation and pedagogic interventions. For example, the business model canvas of startups or the value canvas of NGOs can be analysed for weak spots and sources of potential failure (such as assumptions about citizen behaviour, ecosystem partnerships).

“A resilient society lets you fail, and even lets the failure sting, but only for a moment. Then it helps you get back on track, and everyone reaps the benefit,” Megan signs off.