When human and analytics mix; how Big Data can help with Big Decisions

We have been hearing a lot about the use of Big data and how it can make it easier to take informed Big decisions. But, the connect between the two is far more complex than that. This can best be illustrated with an example.

Let’s assume you are a person with a passion to cook good food and have opened a restaurant. People liked your food and also your interactive approach and your restaurant becomes popular. Without realising it, you are mentally mapping the age and food preference of every customer and make sure everyone is delighted when they leave your place. When they get what they wanted, all of them are happy. Some of them who come to consume food travelling miles ask you for a new branch. You start the second branch, hire more staff and your business is growing to a large scale.

10 years pass on. You now have 50 branches and open 3 or 4 new branches every year. As your business is doing well, you started a training academy to recruits trainees and train them as chefs in your branches. You personally teach some courses to ensure customer delight, quality and taste. In short, you are now a Big company.

Thanks to computerization across branches, you have up-to-date data on daily sales every morning. One of your first customers, now a personal friend, advises you on strategy. He is a multi-disciplined personality who worked in multiple business area around food. You still go to him for help to take decisions. You trust him more and talk to him in the path towards decisions. He is your strategist.

Now, your business executives help you identify location and rent a place and also suggest the investment required for each location and the ROI expected. They provide you with multiple options and ask you to select a location.

You need to make a decision “Where should you open the next branch? Which city? Which location?” These are big decisions and you are wary as the risk is now higher.

You turn to your two key executives, your Sales head and your Operations head. Both of them joined in the early stages and are good at their functions. They have set up processes to collect data that they perceive to help in decision making. You can also get other data from your IT manager.

Now here’s your problem.

• Your sales and operations heads are good in their own areas but have little understanding of each other’s’ functions.
• Both of them have come up by doing their job well. But they aren’t experienced enough to help with overall decisions and the implications of big decision making.
• The IT manager understands the systems and databases but doesn’t know enough about the business to mine the systems.
• Your strategist friend is a computer novice and has little understanding of databases and systems that can help the business.

The question now is how do you go about making the Big Decision of where to start your next restaurant? Let us see how Big Data can play a role.

First, Big Data does not necessarily mean appropriate data or the right analytics approach for a specific scenario. Second, one cannot always rely just on data analytics alone and ignore the human element. Thus, in reality, experience and intuition, data and analysis are not mutually exclusive. Can Companies we find right people with expertise in each of the above areas and ability to marry all the four together. Hence the real question is how do you marry the two and reach a decision?

In this case, you would probably get the strategist to ask questions that he as a food industry expert would know to ask, while the operations and sales heads will provide their inputs to this along with the business owner.

While this is a practical solution, it is not really the best use of Big Data. For example, no one is qualified to seek data that spans across departments, the IT manager, who thinks in terms of SQL queries and department based data may not present it in a format that makes big decisions easier.

Time is critical as the decision is big and the stakeholders are big. Decisions cannot wait for the Big Data to come. Also remember that when the strategist views the report, they might want to drill more or they might even throw the data seeing its insignificance to the business or identifying the fallacy in their hypothesis.

So, at the end, some data, some human experience and a lot of intuition will go into this big decision. Our objective is not to minimise any of them, but to arrive at the best possible combination of the three.

One big problem in making big decisions is the focus on specialization. Today, everyone is working towards getting their department to perform well, and the willingness to experiment is going down. That, in my view, no interconnect between departments is the biggest hurdle to making informed decisions using Big Data.

As companies get larger, problems get more complex and stakes get bigger and even this approach may be considered too simplistic. Problems are muddy. Problems are interconnected. We do not know what problem to solve. Lots of times, the solution may not be available internal and might available as part of industry market research data.

This calls for a change in company culture. What should organization to do motivate engineering managers about value proposition of the product rather than product or technical features to make smart decisions with related to buy or build a feature? How should organizations enable product managers to understand customer support issues to come with higher priority for customer needs? Do you have a policy to rotate potential employee across different functions to enable them to learn from each other? Executives should be enabled to learn from each other, experiment to make big decisions frequently, apply decision and learn whether decision works or not for business. It not, they need to equip with learning to take new decisions. In addition they need the learning instinct on a continuous basis.

Hence in addition to Big Data, we need people with skills to bring diverse data together to make decisions better, collaborate and share stories with other department experts to help them relate to find superior solutions, propose experiment hypothesis that can be rejected by different department based on the nature of risk and mitigation measures needed and continue to ask stupid questions that helps them to keep learning and validate the past experiences based on current learning.

With this type of challenge to Big decisions, we see that Mu Sigma is positioned ahead of curve with focus on creating ‘decision scientists’ rather than ‘data scientists.’

To start with is the core belief of their approach: “All business is about understanding how their customers and others are taking decisions. And if I can understand that about people, I can influence their decision-making process,”  They have proposed 3 axioms of Mu Sigma to guide fresh engineers.

• Learning was becoming much more important that knowing. This needs a structured test and learn approach
• How people learn was becoming more inter-disciplinary
• Extreme experimentation – throwing darts randomly and hoping to hit bull’s eye. This has potential to create large impact on company’s future.

Where do we find these decision scientists? They need to be developed and that needs a learning pedagogy to be followed. We see Mu Sigma has aptly chosen the Montessori mode of education in HBR article “Develop Leaders the Montessori Way,” to bring these axioms to reality in corporate culture. Lot of parents and teachers  know that Montessori model emphasizes on independence, freedom within limits and respect for a child’s natural psychological development.

• It encourages children to be curious to create their own game experiment and allows them to choose some of the existing   game experiments and allows them to refuse to perform some game experiments without stating any reason. In a company setup, this means employees are motivated to move beyond how of the scenario and learn what and why of the scenario. They are not driven to work hard alone (perceiving promotions as carrot).
• There is no impact in teacher behaviour whether you win or lose the game. In company experiment, success and failure needs to be treated the same and both needs to be provided with right guidance and encouragement. In the corporate setup, employees are enabled to have a long term vison for their position with the company, which serves also as a genesis for their career, rather than just an entry level job.
• The Montessori model appreciates diversity of skills. Teachers do not consider that knowing alphabets and numbers is needed more than being neat and hygienic and want the child to grow holistically. Corporate also need to have heterogeneous teams as they are more creative than homogenous ones. When this happens in the company, Diversity yields divergent thinking. There is a good possibility that pooling of a broader knowledge results result in learning that leads to better strategic choices.

Do other companies other than Mu Sigma take the approach of Montessori method of education? Yes. Some of the top companies are already following these principles. Hindustan Unilever or Tata Administrative services train their management staff and rotate people across different functions and brings versatile spice to their already enriched profile . Can this be scaled? Some start-ups are trying to follow this, giving ownership and responsibility / quality and time sense comes as a by-product.

That is the question Mu Sigma is trying to answer. Mu Sigma is training the engineers to think with freedom and interested in “Why of things” and are encouraged to continue this new form of thinking from his new box or new culture. Over years, some of these engineers would grow as decision scientists who are capable to work with other departments to create influence in organizations leading to collaboration and experiments.

Some of these trained people shall be able to connect the dots of diverse data using new technologies, processes and skills for other business and help the business not to  risk drowning in Big Data. Some of these trained people might join other companies bring the learnt culture that would help larger industry to look at big decisions differently.

The industry needs to be changed and not just one company. 

More than just developing the machine ecosystem where data of other department moves using computers and the data gets collated in to Big Data, the companies also need to encourage culture where employees interact with each other and also understand why things are done in specific manner. That means the employee needs to not just focus on what needs to be delivered, but also understand how an individual output enables other individuals or other departments to deliver their results better.

When a company trains employees at entry level with this style of education, the culture is one where employees start to question status quo, irrespective of whether doer is person or machine and also get right answers, it make them realize that there is a larger sense of purpose they start to realize and get satisfactory answers

Only then will Big Data be truly effective in a larger way.