Ziroh Labs pushes CPU-first approach to enterprise AI inference

Bengaluru and California-based Ziroh Labs has launched Kompact AI, a CPU-first artificial intelligence platform aimed at enterprise workloads where the priority is not maximum computing power, but the ability to process large volumes of transactions efficiently. 

The company’s argument is straightforward: as AI moves into everyday business processes such as software testing, document handling, customer interactions and cybersecurity, not every task necessarily needs a large GPU-based system. 

Enterprise AI is becoming more varied as alongside large models used for demanding generative AI applications, companies are increasingly deploying smaller models for routine inference, meaning the stage where a trained AI model processes new information and produces an answer. 

Ziroh Labs said these workloads can be influenced as much by latency, data movement and transaction volume as by raw computing capacity. 

Hrishikesh Dewan, co-founder and chief executive officer of Ziroh Labs, explained the company's approach around broader access to computing. 

“The next phase of AI adoption will require infrastructure that is accessible, efficient, and closer to the people and businesses using it. Our focus is to make AI more accessible by enabling organisations to run AI where it is needed, with greater control over cost, performance and data,” he noted.

That proposition comes as the AI infrastructure market itself becomes more diverse. The industry is not moving uniformly away from GPUs, which remain central to demanding AI workloads. 

AWS, for example, has expanded its GPU infrastructure for generative AI inference, while also publishing guidance in 2026 on optimising CPU-based inference with Intel Advanced Matrix Extensions. AWS said many organisations find CPU inference suitable after considering cost, operational complexity and infrastructure compatibility.

The broader direction appears to be towards mixed infrastructure instead of a simple GPU-versus-CPU contest. Intel and Google said in April that CPUs and infrastructure processing units remain important in modern heterogeneous AI systems, while Intel and SambaNova announced an architecture combining GPUs, specialised processors and Xeon CPUs for different stages of AI inference.

In India, this is unfolding alongside a major push to expand access to AI compute. The IndiaAI Mission, approved with an outlay of Rs 10,372 crore over five years, originally targeted more than 10,000 GPUs. Government data published in 2026 said more than 38,000 GPUs had been onboarded for affordable access to startups and academia.

Kompact AI is also being positioned as an ecosystem rather than only a software runtime. Ziroh Labs said its packaged “Kompact AI Box” solutions are being developed for sectors including healthcare, education, retail and fintech, while partnerships and academic Centres of Excellence are intended to broaden access to practical AI infrastructure. 

The company’s emphasis on keeping AI close to organisational data also has added relevance as India’s data governance framework matures. The Digital Personal Data Protection Rules, 2025 were notified in November 2025, giving effect to the 2023 law and establishing obligations around the lawful handling and protection of digital personal data.

There is also an educational and research dimension. Ziroh Labs said it is working with IIT Guwahati on a 40-plus-hour AI course and on a Math Lab examining the mathematical foundations used in large language and state-space models. 

Prof. S. Sadagopan, founder director of IIIT-Bangalore, linked the approach to India’s broader need for efficient computing. “Frugal innovation and building energy-efficient AI for everyone is what excites me about the platform which is a timely development for India,” he said, while also highlighting Ziroh Labs’ claimed network of partners, technical papers and models.