AI creates value only when employees can effectively integrate it into everyday workflows, says IBM’s Rahul Rao

Semiconductor technology is the bedrock for AI. Unlocking AI’s full potential requires a combination of chips, software, memory architecture among other things.

However, tech is only one part of the equation. Enterprises also need a strong governance layer, workforce readiness and infrastructure readiness before deploying AI systems. IBM believes as AI goes deeper into an organisation, they also need a flexible tech architecture that allows for the workloads to run where it makes both operational and business sense.

In an e-mail interview with EnterpriseStory, Rahul Rao, Distinguished Engineer- Processor Design, IBM ISDL noted that the coming in AI does not mean wholesale replacement of existing tech systems as modernising certain critical parts can deliver the results.

Rahul Rao did his Ph.D in Electrical Engineering from University of Michigan and has spent over two decades at IBM.

“As AI workloads become more diverse, organisations must balance performance, energy efficiency and cost across the entire computing environment,” said Rao.

Edited excerpts:

EnterpriseStory (ES): AI today involves the seamless convergence of multiple elements. What should enterprises do to maximise the benefits of AI?

Rahul Rao (RR): AI success today depends less on deploying a model and more on building the right foundation around it. Enterprises need to bring together data, infrastructure, governance and talent in a way that supports long-term scale.

The starting point is identifying critical data and preparing it for AI through better quality, accessibility and governance. As AI workloads span on-premises, cloud and edge environments, organisations also need flexible architectures that allow workloads to run where it makes the most operational and business sense.

Strong governance is equally important. Enterprises should establish frameworks for security, responsible AI, model lifecycle management and risk oversight from the outset. Workforce readiness is the other crucial element. AI creates value only when employees can effectively integrate it into everyday workflows. This is particularly relevant in India, where initiatives such as the IndiaAI Mission are helping accelerate AI adoption, digital infrastructure and skills development. IBM's IndiaAI study found that 77% of organisations identify accessible, affordable and secure cloud infrastructure as a key barrier to AI readiness.

ES: What challenges do enterprises face in integrating the hardware, software and system elements required for the new era of AI workloads?

RR: AI workloads are fundamentally changing how enterprises think about infrastructure. Training and inference place very different demands on compute, memory, networking, power and cooling, creating new levels of complexity across the technology stack.

At the same time, the semiconductor industry itself is evolving. Performance is no longer determined solely by transistor scaling. Increasingly, gains are being driven by advances in system design, packaging technologies, memory architectures and tighter hardware-software co-optimization. As AI workloads become more diverse, organizations must balance performance, energy efficiency and cost across the entire computing environment.

The biggest challenge for enterprises, however, is integration. Most organisations operate a mix of legacy systems, multiple clouds and diverse hardware environments that were not designed for AI workloads. Success increasingly depends on ensuring these technologies work together seamlessly rather than optimising individual components in isolation.

ES: Does this mean enterprises need to overhaul their existing infrastructure to meet the demands of AI workloads?

RR: Not necessarily. Most enterprises do not need a wholesale infrastructure overhaul. Instead, they need a targeted modernisation strategy that builds on existing investments while addressing bottlenecks that limit AI adoption.

Organisations should focus on improving data access, reducing unnecessary data movement and identifying workloads that would benefit from newer compute, storage or networking capabilities. In many cases, selectively modernising critical parts of the environment delivers greater value than wholesale replacement. Additionally, enterprises should focus on having a portable AI runtime stack that supports heterogenous hardware and hybrid cloud architecture to leverage their investment while enhancing utilization and return-on-investment.

This conversation is especially relevant for India as investments under India Semiconductor Mission 2.0 and the broader AI ecosystem will continue to strengthen the country's compute and digital infrastructure capabilities. For enterprises, the opportunity is not simply to deploy more technology, but to build environments that are scalable, efficient and ready for the next generation of AI workloads.

ES: What are the top priorities for enterprises as they enter the AI era?

RR: As organisations move from experimentation to enterprise-scale AI adoption, three priorities stand out:

First is governance. Enterprises need strong frameworks for data, security, model management and responsible AI. IBM's IndiaAI study found that 68% of organisations view governance gaps as a barrier to scaling AI, highlighting the importance of building trust from the outset. Enterprises also need to have evaluation and feedback cycles in place for representing their quality and performance requirements.

Second is workforce readiness. AI adoption succeeds when employees understand how to apply these technologies in practical business contexts. Continuous learning and skills development therefore become strategic priorities.

Third is infrastructure readiness. AI workloads are becoming increasingly complex and resource-intensive. Organisations need infrastructure that balances performance, efficiency, security and cost while supporting long-term growth. The leaders will be those that can align technology investments with measurable business outcomes rather than isolated AI experiments.

ES: How is IBM helping enterprises drive this convergence?

RR: IBM's perspective is that the future of AI will be shaped by convergence across semiconductors, systems, software and hybrid infrastructure.

Our work spans semiconductor research, chip design, systems engineering, hybrid cloud and AI software, providing visibility into how these technology domains are evolving together. A recent example is IBM Research's demonstration of the world's first sub-1 nanometer chip technology, which pushes the art of possible in semiconductor technology.

More broadly, we believe AI is driving a shift toward system-level thinking. IBM is enabling fully governed and client-controlled deployment of AI and bringing AI to their data within their trusted environments. IBM’s recent processors and Power and Z systems enable on-chip and on-system AI accelerators to provide enterprises exactly that.

Enterprises are increasingly evaluating infrastructure based on how effectively data, compute, software and governance work together to deliver outcomes. Our role is to help organisations navigate that transition by combining insights from AI, hybrid cloud and semiconductor innovation, while enabling them to build secure, scalable and trusted AI foundations.