Success tips for Big Data: culture of analytics, data scientists and customer focus

Thomas H. Davenport is the author of BigData @ Work: Dispelling the Myths, Uncovering the Opportunities (see my book review). He is Professor of Information Technology and Management at Babson College, Cofounder and Director of Research at the International Institute for Analytics, and a senior adviser to Deloitte Analytics.

Davenport has spoken at a number of countries around the world on the use of analytics by business leaders, including NASSCOM’s Leadership Forum 2014 in India. He joins us in this exclusive interview on the connections between Big Data and knowledge management, challenges for product startups, data gaps and decision making.

YS: You have written some of the pioneering books about knowledge management (KM). How would you connect the fields of KM, Big Data and analytics today?

TD: That is an interesting question. These fields are definitely related, though they may be on different branches. Analytics can be interpreted as knowledge derived from data, whereas KM focused more on knowledge derived from collaboration and information. Few have made the connections between these fields and harnessed them in a combined manner.

YS: What is your current field of research in Big Data and analytics?

TD: I am looking at two areas: companies that create new products from Big Data, and also at what automated systems and decision analytics do for knowledge workers - in terms of augmenting their intelligence as compared to automating their actions and decisions.

YS: How big a role does academia play today in advancing Big Data research and practice? 

TD: There are some degree-oriented programs and majors in educational institutes which address Big Data, but there is not much connection yet between academia and industry in this field, which is unfortunate.

YS: How was your book received? What were some of the unusual responses and reactions you got?

TD: In my book, I end by describing the new era we are entering as Analytics 3.0 - prescriptive analytics, going beyond the first two versions of analytics (based on descriptive and predictive analytics). However, one reader wrote to me that this is actually Analytics 2.1!

Other than that, I think the book has been well received since it is perhaps the only Big Data book focused on how large companies use Big Data for business decisions.

YS: How are governments using and regulating Big Data?

TD: On the one hand, we are seeing lots of activity by government agencies in Big Data, in sectors such as intelligence. Governments are also active in Open Data, and we are seeing many national and local governments using Open Data in sectors such as public transport, eg. predicting bus arrival times.

On the other hand, we are also seeing many governments cracking down on data breaches and privacy violations by companies. There are a number of regulations in Europe, for example, on use of data in marketing. Even without such regulations, the key for companies is to gather and use Big Data without making customers unhappy.

YS: What are the typical challenges Big Data startups face as they scale up their company?

TD: It is becoming increasingly easy to develop Big Data products – but getting customers to pay for them is another matter! Ad oriented models will have limits. Startups also face shortages in hiring and retaining data scientists. Another challenge is doing analytics well – going beyond data to understanding, predicting and adjusting to customer behaviour.

YS: What trends are you seeing in VCs investing in Big Data and analytics companies? 

TD: I don’t have solid metrics on this, but I do see Big Data and analytics as key focus areas for investors, though some are exercising caution on choice of sector and portfolio.

YS: In the coming years, will there ever be a ‘computational cliff’ or ‘management abyss’ beyond which there is just too much data to gather and interpret?

TD: This is already the case! It is estimated that companies are acting on only half of one per cent of the data that is available. We are already behind the data wave in terms of scale and speed. Phenomena like the recent ‘flash crash’ where stock prices fell precipitously in a short time reveal that the financial services sector is struggling to keep up with data signals.

YS: What change management is needed in an organisation to create the Big Data culture you describe?

TD: Many companies are introducing Big Data culture by bringing in new executives who have domain experience as well as a data scientist background. Companies have to change by increasing their sensitisation to the potential of Big Data, acting on the opportunity, and driving change through champions.

YS: How should SMBs (small and medium businesses) be approaching Big Data today? 

TD: Many new online startups already have the Big Data culture built into their DNA. The challenge is for older SMBs, especially those operating in the largely offline world, to build Big Data awareness into their culture. The smarter SMBs are watching technology changes and seeing business trends.

YS: How should innovators strike that delicate balance between ‘Stick to your vision’ and ‘Adapt to a changed world’?

TD: Analytics will not really tell you if your world has changed – executives have to rely on their intuition, partner dialogue and competitive assessment to see if the world has changed and how they should switch strategy. For example, Caesar’s resort may be good at using analytics to come up with new gambling offers and charges for customers – but a decision on expansion to Macau will come from other considerations.

YS: In the time since your book was published, what are some new successful case studies you have come across in Big Data usage? 

TD: Since my book was published, I have come across companies like Monsanto in this space, who have acquired weather information companies to offer precision planting services for its customers based on Big Data patterns. In the financial sector,

Barclays and JP Morgan Chase are harnessing Big Data. Another major frontier is Internet of Things (IoT) – I am seeing the rise of Analytics of Things (AoT).

YS: What is your next book going to be about?

TD: My next book is coming out in March 2015. It is about how knowledge workers can add more value in a scenario where more and more decisions are powered by Big Data. How should doctors, teachers and lawyers adapt to a Big Data world?

YS: What is your parting message to the startups and aspiring entrepreneurs in our audience?

TD: Make sure you have all the Big Data you need to make decisions. If you don’t have that foundation of data then you have to fall back on your intuition and guts!