Analytics and AI: digital transformation tips for the algorithm age

This new book brings managers up to speed on the business potential of analytics and artificial intelligence (AI), and how to gear their organisations up for change and innovation.

Startups and enterprises looking to harness the power of exponential technologies can find useful checklists, frameworks, and case studies in the new book, AI and Analytics: Accelerating Business Decisions by Sameer Dhanrajani.

“AI is the natural evolution to changing genres of sophisticated analytics, further strengthened by an algorithm economy,” Sameer begins. “AI is rapidly becoming the vehicle to address and solve the most pressing societal problems related to healthcare, security, education, and allied areas. This makes it akin to the next Industrial Revolution itself,” he adds.

Sameer is the Chief Strategy Officer at Fractal Analytics, and was earlier at Cognizant Technology Solutions and Genpact. The book is drawn from his 150 blogposts over five years, but the book could do with a lot more editing to remove repetitive material. The painfully small font makes it difficult to read across the 372 pages, and many figures and charts are illegible.

Here are some of my key takeaways from the book in terms of trends, impacts, organisational change, startup players, and future challenges. See also my reviews of the related books Big Data Revolution, Big Data @ Work, and Life 3.0.

Trends

The book begins by identifying key trends in this space: industrialisation of analytics; data collection and monetisation; analytics-led business transformation; and rise of the algorithm economy in areas like behavioural prediction. Market forces drivers for analytics are technological advancement, rise of consumerism, data explosion, and competitive pressure.

Depending on the level of maturity with respect to reactive, tactical, and strategic use of analytics at the local or enterprise-wide level, companies can be classified as laggards, amateurs, practitioners or masters. Analytics in mature organisations is business-user friendly, and grows beyond the purview of CTOs and CIOs. In a data-driven economy, data sits at the core of every business model, leading to real-time insights and agile decision-making.

AI is powering new models of assisted intelligence (in clearly-defined, rules-based, repetitive tasks, for example in Gmail), augmented intelligence (for example, clinical decision support; Netflix recommendations based on individual and group patterns), and autonomous intelligence (for example, facial recognition, driverless cars, automated trading).

Transformation 1: Analytics

The journey to analytics transformation is not a sprint but a marathon, Sameer cautions; it involves technology, process, culture and leadership change. Just gathering lots of data is not enough; many companies have large volumes of “dark data” in silos.

The transformation calls for unification in approaches across divisions and functions, internally and externally. Companies should think big, frame the right questions, craft a data agenda with effective quality and governance, and embed insights at the right touchpoints to deliver immediate value, Sameer advises.

Effective “data horsepower” depends on data quality, storage, assimilation, sanitisation, harmonisation, and visualisation. Data readiness should be followed by technology and business readiness. Security and privacy of consumer data should also be respected.

Close cooperation is needed between the data management team, data modelers, data scientists, visualisation experts, domain experts, and business unit heads. Early initiatives can focus on projects and tactics, then scale up to strategy and partnerships.

In the age of the predictive enterprise, the “data deluge” should be converted to a data monetisation engine. Data is worth monetising if it can predict behaviours, lead to actionable insights, refine understanding of existing and potential customers, and make a decision with precision.

Sameer offers a 2X2 matrix comparing data maturity and analytical maturity to decide whether corporate decision-making is gut-based, reactive, tactical or forward-looking. Speed and quality of response are important if they can prevent fraud or failure, reduce customer waiting time, and improve resource utilisation.

Decisions to build, buy or outsource analytics solutions and services should be taken based on criticality, differentiation, availability of in-house talent, and vendor considerations of ease of use, cost-effectiveness, and granularity of control.

In additional to transactional Big Data about consumer activities, companies need “thick data” about consumer behaviours and emotions via qualitative insights. This helped Lego understand, for example, that children wanted not just quick gratification but the longer experience of imagining and creating.

“The problem with Big Data is that organisations can get too obsessed with numbers and charts, and ignore the humanistic reality of their customers’ lives,” Sameer rightly cautions (see my review of the related book Sensemaking).

Such insights help understand the “choice architecture” of consumers, and use methods like “social proof” to induce change via “behavioural nudges.” Such techniques are effective during election campaigns, for example.

Transformation 2: AI

Innovative giants such as the FANG companies are rapidly disrupting a number of sectors with AI. Robotic Process Automation (RPA) is disrupting IT services and BPO companies. A company should make itself AI-ready by embedding intelligence in processes and platforms, fostering a learning culture, encouraging innovation, deepening internal relations, re-skilling teams, and preparing for tough ethical challenges.

Engaging with tech experts, consultants and startups also helps. The focus should be on processes and not just departmental functions or silos. “Algorithms can work at a speed and scale that cannot be easily matched by scaling the human workforce,” Sameer explains. Robots and algorithms can also build other robots and algorithms.

Business leaders must figure out how to design and leverage algorithmic business models (“the economics of connections,” as described by Gartner). “Mr Algorithm” may well be the new member in the boardroom, Sameer jokes. “To some degree, every company will become a math house,” he adds.

Inputs from fields like design thinking will become increasingly valuable, by providing human-centred creative angles. Companies should set up a design research capability, to combine context with analytics. This helps with problem reframing, structured ideation, and responsible and inclusive design; it applies to the consumer as well as internal enterprise contexts.

Careers and skills

Analytics practitioners need domain knowledge in statistics and machine learning, skills in storytelling, experience in data engineering, and a sense of academic curiosity. “The data must tell a story that can be understood by all stakeholders,” Sameer explains.

Visual dashboards play an important role here as well. The data scientist’s role goes beyond collecting data and developing products; it is also about engagement and quality. Companies will need a culture of fact-based decision-making, powered by data science expertise and AI strategy. Success in this era even calls for being comfortable with uncertainty and acting with agility.

The rise of AI will make it all the more important for humans to reposition themselves towards tasks requiring judgement, creativity, empathy, ethics and regulation. This could also be the time for a career pivot for traditional IT professionals, who can branch out into new areas like Blockchain and cybersecurity. Adaptability is key to unlearn and relearn, Sameer advises.

Business impacts

AI is well-suited to tackle complicated and complex problems, as well as tasks with drudgery, risk and repetition. Analytics and AI are transforming operations, HR, finance, marketing, and strategy. They reduce decision latency, strengthen fire-fighting, improve risk modelling, refine product categorisation, and speed up responsiveness in real-time scenarios involving airplanes, power plants, and self-driving cars.

Impact areas are across sectors: banking (fraud mitigation, customer retention, risk management), automotive (connected cars, drones), insurance (digital actuarian models), healthcare (at the level of patients, drugs, treatments), retail (personalisation, supply chain optimisation, omni-channel commerce), and cybersecurity (monitoring and predicting attack patterns).

“AI has been critical in research and development to eliminate human error, which is expected to have a major impact in fintech,” Sameer explains. Services can also be personalised to a “customer segment of one.” Robo advisors can help users with goal-based planning and wealth management.

Other “man-machine learning” (MML) scenarios arise in automated and augmented underwriting in the insurance sector, such as claims management and fraud prevention. Smart wallets will guide users to more responsible spending habits. Better models can be used for collection services and punitive actions.

Analytics in healthcare can be used at the level of medical records, diagnosis, treatment, and follow-up care. Real-world evidence (RWE) helps track differences and similarities across comparator groups, which can be used in designing disease treatment. Detailed datasets and analytics for health technology assessment can improve clinical trial design, for example the use of bio-markers.

Analytics can also yield rich insights from patient portals such as PatientsLikeMe. AI is a “boon to the life science industry,” Sameer explains. Sectors like genomics show how coordinated approaches towards data processing are the future of the industry. Genomics can help identify high-risk patients in diseases like diabetes.

AI can also be used to manage regulatory compliance, for example misinterpretation of rules. Open source tools are emerging to explore large genomic data sets, for example Integrative Genomics Weaver (by MIT and Harvard). The key is to manage large volumes of data in real time and find pertinent correlations quickly.

“Predictive analytics has been used in retail for decades; however, in the last few years, AI with other advances in technology have super-charged the speed, scale and cost at which it is used,” Sameer explains; the accuracy has also increased.

Diversity in retail options and the rise of mobile and social media have given consumers more choice and voice than before. “A data-rich industry, e-commerce benefits widely with AI adoption,” observes Sameer, pointing to intelligent product discovery via visual and voice search as an example.

Customer expectations have risen, and they expect to be rewarded for their loyalty; they want exclusive, personalised, and relevant offers. Businesses will in turn need to get smarter to gather valuable data, assess deep context instantaneously, harness collaborative filtering, reduce customer churn, and detect and reduce lost sales.

“The application of algorithms is witnessing a shift from productivity based in the back end to a core selling model in the front end,” says Sameer. “To be effective, customer intelligence needs to include raw transactional as well as behavioural data,” he adds.

Attribute analysis and event sequence analysis are useful computational techniques in this regard to get a more complete view of the customer. In omni-channel retail, customers expect consistency, agility and ease at all times.

Analytics and AI are creating fluid supply chains; smart logistics enables better demand planning and forecasting. Inventory tracking, route optimisation, product placement, and promotional displays are other areas where analytics makes an impact.

CXO impacts

Impact areas of analytics and AI also span different functions and managerial roles, such as marketing, advertising, HR, finance, and enterprise information management. For example, the CMO will be able to improve engagement marketing at scale in real time, harness contextual insight-driven automation, and break out of silos to enable effective cross-selling and upselling.

Customer churn can be reduced, and loyalty and win-backs tracked in real-time. Social media listening and rich data flows within the organisation can improve brand perception analysis, and predictive powers will be enhanced on a scale not previously possible, says Sameer. Decisions based on such actionable insights will be smarter, faster, repeatable and replicable.

“HR is shifting from being an art to a science,” explains Sameer, pointing to the use of chat bots, employee sentiment analysis, and fraud detection. The rise of talent sciences represents a new era in HR, by enhancing human capital management via better psychometric and behavioural data analysis.

This includes pre-hire selection methods beyond gut instinct, training based on learning analytics, job rotation, attrition prediction, and succession planning. Gartner research classifies HR analytics into four levels: workforce summary, HR processes, employee performance, and workforce planning.

CFOs will be able to dig below the numbers to their source along with better business correlations; this can improve invoice clearing and expense claim auditing. The rise of IoT and Industry 4.0 frameworks will create new types of CIO leadership persona: explorer, ambassador, clarifier, educator, attractor and cartographer, according to Gartner.

Challenges

A range of challenges is holding back the rapid rise of AI. These include lack of understanding about AI, absence of standardised and structured frameworks, inadequate data awareness or strategy, inability to vision or morph to new business models, and lack of re-skilling to make workforces AI-ready.

“Enterprises must figure out how humans and machines can collaborate and complement each other to create competitive advantage and synergies,” Sameer urges. This also applies to risk management, where AI can help companies monitor and respond to cyber-attacks and fraudulent behaviour based on pattern analysis at the level of devices, processes and networks.

The author rightfully identifies a range of other business, ethical and legal challenges that companies need to factor in as they increase their capabilities in analytics and AI. These include customer and employee privacy, enterprise change management, and upskilling to deal with work transformation.

Case studies

The book is packed with case studies and profiles of sectoral business impacts. A data focus can create “new business moments,” such as Google leveraging previously untapped hyperlinks as a data resource; AI now drives everything from its search platforms to driverless cars. A significant chunk of business at Amazon comes from its recommendation engine; the company’s other initiatives also represent the state of the art in deploying an “integrated strategy machine.”

Nike has integrated data to avoid potential stock-outs and maintain optimal inventory levels. Data monetisation at Vodafone Netherlands led to stronger marketing success by improving relevance of targeting and market positioning.

Tesco Bank uses ClubCard customer data to identify customer needs and create new personalised products and services. Freight company Pitt Ohio uses predictive analytics to increase repeat orders and reduce the risk of lost customers. Analytics helped PPL Electric improve its service reliability metrics.

Data collection and open reporting have been used for public works monitoring in South Africa (WaterWatchers), health services tracking in Uganda (Cipesa), spotting illegal fishing structures (Iran), comparing educational policies (World Bank), and modelling agricultural risk in Kenya (GeoVentures),

Emcien delivers its pattern-detection capability via cloud-based analytics-as-a-service to retailers and telcos. Host Analytics, 8thBridge, and Dachis Group offer insights-as-a-service, drawing on customer data and social media. JBara offers business best practices for improving customer profitability, while 9Lenses helps companies benchmark their performance.

The Talla chat bot assists in interview question management. TextIO uses AI to detect biases in communication during interviews. HireVue helps companies analyse interview videos for cues from voice inflections, micro-expressions and verb choices. BluVision radio badges track employee movements and flag intrusions or time-wasting behaviours.

Minhondo improves employee networking by profiling their workspace and online browsing activities. Bain has researched how companies track and improve efficiency of meetings to reduce wastage of time.

JP Morgan uses algorithms to track and reduce rogue trading among employees. JP Morgan Chase has a contract intelligence platform (CoIn) to analyse legal documents. American Express uses predictive analytics to identify at-risk customers. Lemonade uses AI to speed up its home insurance signup process. Progressive uses telematics to track driver performance and customise a range of car insurance plans.

Philips IntelliSpace uses AI to improve performance in ICUs. Moorfields Eye Hospital is using Google Deep Mind Health to track eye disease complications. Texas Health has partnered with Healthways to identify high-risk patients and offer customised intervention. Pfizer uses Watson for Drug Discovery to mine research articles for insights on better drug usage in cancer treatment.

Healthcare company Merck created a data-driven solution called MANTIS (Manufacturing and Analytics Intelligence) which helped reduce inventory carrying costs; rollout began with a “lighthouse” project in the Asia-Pacific, followed by a broader rollout.

Flipkart’s Project Mira uses AI to query users about what they want. At SnapDeal, 40 percent of total sales are driven by predictive algorithms. Pandora and NetFlix leverage hyper-granular behavioural profiles of customer shopping habits at scale.

Alibaba’s TMall uses robots in its warehouses, and DHL uses autonomous forklifts. UPS improved its route schedules and customer service through its On-Road Integrated Optimisation and Navigation (ORION) system.

WalMart blends offline and online shopping via in-store Pick-up Towers for online shoppers; facial recognition is also used to identify unhappy or frustrated customers at checkout lines. Macy’s has leveraged Big Data for real-time visibility and pricing, by drawing on location and demand insights. Toy retailer Entertainer used micro-segmentation analytics to increase digital channel sales. LinkedIn uses the “also viewed” feature to show profiles and jobs to those already viewed.

Disney’s Magic Band bracelets yield valuable insights in consumer persona while helping visitors in park navigation, ticketing, and room bookings. Knorr, through a social media campaign called #WhatsForDinner, uses AI to recommend recipes based on what consumers have in fridges.

GoFind uses a deep learning-based image search engine to help people find products in online stores. L’Oreal and CoverGirl use ModiFace’s facial modelling techniques to help with product discovery. TheTake uses AI to help people find products similar to what they see on TV. Google Assistant helps voice-based search of images on smartphones.

Siemens is using the Industry 4.0 framework for automating supply chains and making factories self-organising. BMW ID profiles help leverage data in the connected car ecosystem. EverScreen uses AI with security cameras to track errors in checkout scanning.

The Associated Press is generating reports with AI-powered robots, and focusing journalists on more investigative reporting. Pinterest uses a combination of AI and crowdsourcing to help users find what they want through better curation.

Airbus used fuzzy matching and self-learning algorithms to reduce production disruptions. BP augments human skills with AI to improve field operations. Insurance company Ping uses AI to speed up its loan offering capabilities with better customer scoring capabilities.

The US Postal Service uses deep-learning neural networks to automatically read handwritten zipcode digits. Other high-profile AI stories are exemplified by AlphaGo, Watson, and Big Blue.

Startups

Large companies such as Google, Microsoft, Amazon, Facebook, NetFlix, LinkedIn and GE are leading the AI wave. A number of startups have also entered the fray, such as SearchLight Health (healthcare sector analytics), Netradyne (determining causes of automobile accidents), Node (natural language processing), Dumbstruck (emotion analysis), Shift Technology (claims management), ControlExpert (auto insurance), Libra (genomics), Layar (AR), and Premise (consumer price index). New devices have also emerged in healthcare, such as Asthmapolis (GPS-enabled tracker to record inhaler usage).

CB Insights classifies AI startups in fintech into segments such as credit scoring, direct lending, personal assistants, asset management, fraud detection, insurance, sentiment analysis, debt collection, reporting, and predictive analytics. “For traditional banking institutions, the focus and energy for innovation are simply not there, nor are the necessary IT budgets,” Sameer observes; this has driven many corporates to engage with startups.

Many large firms have also acquired analytics and AI startups, or made strategic investments in them, for example Ford (Argo), Citibank (FeedzAI), RBS (Lavo), Roche (Bina Technologies), and Google (MoodStocks). Wells Fargo has a startup accelerator programme for the fintech sector. 

The road ahead

Emerging frontiers to watch are IoT (streaming analytics, edge responses, embedded intelligence), Blockchain (usage of new asset classes), and chatbots. “Algorithmic marketplaces will disrupt the analytics ecosystem and likely the whole software ecosystem,” Sameer predicts.

Embedded and connected intelligence will transform the automotive industry, improving in-vehicle content and services while also spawning new business partnerships for road safety and fuel efficiency. A new range of creative possibilities will arise with self-aware automotive vehicles and networks, as AI transforms self-driven cars and drones.

IoT represents new challenges via the need for edge analytics and not just centralised analytics; issues to tackle are performance, latency and security. New frameworks, partnerships and monetisation models will be needed for smart cities, homes, and seamlessly interconnected media and manufacturing sectors. Analytics will also help policymakers understand new frontiers like cryptocurrency, for example spending patterns, transaction traces, and international transfers.

In the long run, though analytics and AI will advance, there will still be decisions that require insights beyond what AI can interpret. “We need to consider AI not as machines, but as colleagues,” Sameer advises. AI algorithms can augment human capabilities, and new business processes can be created based on augmented working strategy.

More exploration and experimentation are called for in this brave new world. Ultimately, the evolution of technology is creating a “new normal,” and will drive humans to find a new purpose in life.