How AWS is leaving the competition behind in the era of IoT, AI and ML

At re:Invent, Amazon launches more than 12 automated services that let engineers scale up deployment of applications on the cloud on a real-time basis instead of old-style manual provisioning.

A couple of months ago, at the Oracle Open World, when Oracle Chairman Larry Ellison claimed that Oracle’s self-governing database would redefine the cloud business and take on Amazon Web Services, the world wondered how.

Numbers did the talking on November 29 as Andy Jassy, the CEO of AWS, showcased the reality of the cloud business today. Amazon is leading the way and is bigger than 14 cloud service providers, including Microsoft, Google and Oracle.

Amazon Web Services (AWS) is on an $18 billion run rate, has achieved a 42 percent growth rate year-on-year and has almost every business working their applications. As many as 3,000 governments, 22,000 non-profits and 8,000 academics are using AWS.

Andy says: “We have thousands of Systems Integrators (Sis) that have built cloud practices, including Infosys and Wipro. Then there are a host of cloud-first businesses building services for the cloud; they all use AWS.”

AWS has launched more than 15 services across Machine Learning, AI, IoT and infrastructure services. All these services democratise IT and free up engineers; they no longer have to worry about managing everything from latency to security.

Other services help a company manage data instances securely, servers and computer that is on-demand and dynamic and back up 100s of TB of data per second.

AWS also launched services like Amazon Elastic Container Service and AWS Fargate, which runs containers without having to manage servers and clusters. It runs on Kubernetes, an open source platform that allows you to deploy and scale apps faster. AWS also allows seamless integration of on-premise solutions with the cloud, which can automatically be deployed across different zones.

“We have automatic patches and upgrades. We are way ahead of the relationship database era and corporations want modern technology,” Andy says.

Amazon is also betting big on data coming from edge devices like cars, machines, agri fields, oil fields and cell phones.

“Today waiting for data is the hardest part; it is delivered slowly because there are so many devices to integrate. People need the cloud for IoT implementation and to make sense of data across devices,” Andy says.

John Deere, for example, has 100,000 tractors working on IoT sensors, sending data from the cloud through the AWS cloud. AWS announced IoT 1-Click, which activates all devices instantly and does away with third-party IoT solutions. The device management service organises and remotely manages all deployed devices.

According to Gartner, AWS has a 44 percent market share in the cloud, up from 39 percent last year.

“We think about builders’ sentiments. It’s about freedom and how builders want to compose their applications. It’s like musicians composing songs; you have to allow them to create with richness and creativity,” Andy believes.

Has AWS really liberated businesses with the cloud?

When builders want to move to cloud they want complete functionality. AWS has an incredibly robust and diverse tech infrastructure business built over a decade. They are in 16 regions with 44 availability zones and 17 more coming soon. Their range of services includes serverless computing, analytics and text and voice AI.

Mark Okerstrom, CEO of Expedia, a travel company that grosses $90 billion in gross travel revenue, is on the AWS cloud. He says AWS helped them migrate to the cloud faster and will help them handle large amounts of data on the fly.

Expedia is a diversified platform with 600 million users. They handle 55 million phone calls and 1.6 million active customers a day. In reality, they are a people logistics company.

“Our search volume is 750 million searches per day and we were not born in the cloud,” Mark says.

Expedia started as a small division of Microsoft 20 years ago. In 2009, they went through a massive code redux and have today written 10 million lines of C++ cloud.

“We sit with 45,000 servers. In two years we will have 80 percent critical apps on AWS. Why are we doing this? We need resiliency, optimisation and performance of data,” Mark says.

Disaster Recovery requires military precision because every company wants zero latency.

“It is all about the core mission, which is about being customer-centric and offering absolute personalisation,” Mark says. No wonder Expedia delivers perfect travel search results and scans data of 3,00,000 hotels in a second to give personalised information.

AWS migrated Expedia to the cloud in 91 days and now allow Expedia’s engineers to manage all this data dynamically.

If Expedia moving to the cloud was a big thing, think about a company whose assets are worth $860 billion, is more than 157 years old and has written one billion lines of code. This company is Goldman Sachs, which moved all its modern mobile-based lending and collaboration tools to the cloud. The company has close to 11,000 engineers.

Roy Joseph, MD of Goldman Sachs, says: “We need strong engineering to enable better risk management and distribution. It is all about data privacy and efficiency.”

Indeed Goldman Sachs has gone all mobile. It launched Marcus, a retail lending application that has disbursed 2 billion loans with an average size of $30,000. Marcus allows people to clear their credit card debts.

They also launched Marquee, which features a portfolio of web services and applications available to the firm’s institutional clients. The feature gives the clients access to the risk management and analytical tools used in-house. The Symphony app is a collaboration tool for secure messaging and collaborations between partner banks of Goldman. All these are part of the Amazon cloud. For a large corporation like Goldman, moving to the public cloud is a very big thing since it needs to manage the security of critical financial data.

“This is why we went with AWS, they not only protect data but also made us more innovative and efficient as a company,” Roy says.

Goldman, which needs a platform for dynamic risk management with more calculations in a dynamic compute mode, spent seven years on its public cloud journey.

“While we work with AWS on the public cloud the entire control is with Goldman. We work on a bring-your-own-key model,” Roy says. Think of it like a safety deposit box in a bank where the bank offers you security for your assets. Goldman went with AWS because of low latency.

Even sports associations are turning to AWS platforms.

Michelle McKenna Doyle, CIO of NFL, says: “The American football league uses AWS technologies to improve the game and crunch 3 TB of data per week.”

They have 180 million fans worldwide and NFL’s Next Gen Stats platforms allow broadcasters and content creators to understand each player and their achievements. They also allow them to predict outcomes of player performance. The data is gathered from individual movements, to the inch, because there are sensors on protective pads that are tracking player movements. A lot of AI is used to crunch this for post-game analysis, enhancing fan experience.

Deep Learning, Machine Learning and AI for the future

Everyone on the planet is keeping an eye on the impact of AI.

In a recent report, McKinsey says 800 million jobs will be taken over by robots by 2030. AI is all about helping the consumer, but what is left to consume if people have no jobs?

AWS is building platforms like Polly and Rekognition for builders to create services that make living easy. The classic case is music where pattern recognitions are important to recommend albums or songs. Say you are listening to Ilayaraja and Pink Floyd on the same day, the software should recommend artistes and albums close to the type of music that you listen to.

Matt Wood, GM for the Artificial Intelligence Practice at AWS, says, “Engineers need platforms that can analyse video, images and text very fast. Engineers want custom modules to use AI that can create insights accurately.”

Amazon’s Sagemaker AI platform allows engineers to specify data sets, their size and memory. The platform then picks the compute power and begins to crunch data. Today, AI platforms are constrained by memory because they have to be manually trained by engineers and it takes a massive amount of time to get up to speed to be commercial.

Sagemaker has 10 different algorithms that allow the engineer to design faster and accurate recommendation engines to be on the cloud. The AI engine looks at millions of rows of data and tweaks the data for accuracy. Sagemaker then allows the right data to be made available to clients. If you’re listening to Pink Floyd’s Dark Side of the Moon album, the engine will recommend similar artists and top-selling albums of the seventies like Fleetwood Mac’s Rumors.

Andy believes it’s critical to “protect your crown jewels”. “That is your data. Be it AI and ML, you should have the right tools for the right job,” he says.

No wonder Amazon remains a world leader in the cloud business.

The worldwide infrastructure as a service (IaaS) public cloud market grew 31 percent in 2016 to total $22.1 billion, up from $16.8 billion in 2015, according to Gartner, Inc. Amazon was the No. 1 vendor in the IaaS market in 2016, followed by Microsoft and Alibaba.

Sid Nag, Research Director at Gartner, says: “The market for cloud services is growing faster than virtually every other IT market today, with much of this growth coming at the expense of traditional, non-cloud offerings.”

“The demand for cloud-based IaaS continues on its path of aggressive growth, and the high growth of IaaS is also driving growth in related cloud markets. While platform as a service (PaaS) and software as a service (SaaS) are also exhibiting strong growth, IaaS is expected to show the fastest growth over the next five years.”

We know cloud is central to everything and AI will create higher-end jobs. But no one knows what the new jobs will be yet. In the era of data, governments will have to work with corporations.

With no escaping the era of data, Amazon seems set to rule the roost with its many offerings.