‘The zero-to-one moment for LLMs is gone’: Zoho AI Director Ramprakash Ramamoorthy

While most SaaS companies are haemorrhaging cash to OpenAI and Google for AI capabilities, SaaS unicorn Zoho Corporationsurfed on a different bet. 

After 22 months of grinding through GPU shortages, hallucination fixes, and five model iterations, the Chennai-based unicorn has something to show for it: Zia LLM, an entirely in-house large language model that keeps customer data locked within Zoho’s ecosystem. 

It has positioned the new LLM as a business-focused model with three parameter variants that keep enterprise data in-house.

“We dogfooded the baseline model extensively—Zoho runs on Zoho. Our employees tested it across products, gave feedback through a ticketing system, and we iterated continuously. It took us nearly two years to build and fine-tune the final version. We’re now on the fifth iteration of the model,” Ramprakash Ramamoorthy, Director of AI Research, Zoho Corporation, tells YourStory. 

Dogfooding refers to the process of companies using their own product internally before releasing it to customers.

At present, it comprises three models with 1.3 billion, 2.6 billion, and 7 billion parameters, each trained separately and tuned for different contexts. The Chennai-based firm claims the models perform competitively against similar open-source LLMs.

Zoho said its two proprietary ASR models for speech-to-text conversion are built to run on low computing power while maintaining high accuracy. The models also claim to perform up to 75% better than similar models in standard tests. 

Going ahead, it plans to add support for more Indian and European languages and will also launch a reasoning language model (RLM) soon.

1402 people loved this story

---

How a Chennai-born researcher helped build ChatGPT—the AI transforming the world

Here are the edited excerpts from the interview. 

YourStory[YS]: What was the idea behind launching Zia LLM? How did developing it from scratch differ from fine-tuning global AI models?

Ramprakash Ramamoorthy[RR]:  Back in early 2023, we launched OpenAI integration for our customers. We didn’t want to stop anyone from using LLMs because we could already see their value in our consumer lives—summarising, paraphrasing, drafting emails, etc. But at that point, we didn’t have the infrastructure or know-how to run these models ourselves. 

We enabled a ‘bring your own key’ model, where customers could bring their OpenAI key and plug it into our system. Later, we expanded support to include Google Vertex, Anthropic, and others, depending on the enterprise relationships our customers had. We wanted it to be plug-and-play. 

By late 2023, we also started hosting models like Llama, Mistral, DeepSeek, and Qwen on our own data centres. That required heavy capital expenditure for GPUs to run inference, but the key benefit was keeping data within the Zoho ecosystem. 

Around July 2023, we began building our own LLM. Llama 1 was our initial benchmark, and then Llama 2. The main challenge wasn’t just the availability of data. There are many public corpora, but curating high-quality, differentiating datasets is difficult. We repurposed our data annotation teams from speech and image recognition to do this, without using any customer information.

YS: What were the challenges the company faced while building the model? 

RR: Compute was the biggest challenge. That’s where our partnership with NVIDIA really helped. In 2023, access to H100s was tough—lead times were about six months. We had to build high-performance computing skills, which was new for us. It’s one thing to build models, but another to run them efficiently at scale within our existing data centre infrastructure—power, cooling, all of it.

Initially, it hallucinated a lot—e.g., summarising an email and including things not in the original text. We had to tune it again and again. That’s why we now have models of different sizes—1.3B, 2.6B, 7B. 

The idea is right-sizing. Yes, a 7B model can do everything a 1.3B can, but it's expensive to run. Compute availability also varies by region—sourcing GPUs is easy in the US, but hard in India or China due to export limitations.

The biggest challenge was dealing with the hardware — procuring it, building the know-how, and running them at scale. That is the biggest learning for us.

YS: When building Zia LLM, how did you decide on the architecture and scaling strategy?

RR: It was a best-of-breed approach. Around the time we started, DeepSeek and others were launching successful models. We borrowed a few concepts from Llama, among other models, but our differentiation was in using contextual signals already present in our systems.

But one differentiation is that all of these were generic consumer LLMs. Architecturally, we started with Llama and Mistral, and later looked into Chinese models. But the key was grounding our model in a business-specific context by leveraging the ground truth from our own ecosystem.

YS: Building context-aware systems has been a focus for many firms. How does Zia incorporate this? 

RR: That was a major focus area. For example, a colleague and I might work in the same company, so our context is similar, but not identical. I don’t want access to their emails or payroll data. Similarly, if you're from a different company, your context will be entirely different.

Zia LLM is built on top of Zoho’s search platform. Context is determined by what you have access to—your emails, CRM leads, documents, etc. It doesn't pull in unrelated data. We also built tool-calling capabilities so that if the model needs to fetch something outside its context window, it does so through governed, auditable APIs.

YS: How do you position Zia LLM against other SaaS players investing in AI? What’s your USP?

RR: We’re not a model service provider. We don’t want to be ChatGPT or Gemini. We’ve observed that many SaaS players started building their own LLM efforts in mid-2023. Many of those efforts are now on hold. Even projects from Salesforce or SAP—if you check GitHub, the last activity was months ago.

The open-source model boom has played a role in this. Today, you have 9,000+ models on Hugging Face with various architectures and capabilities. So the question still remains: why build your own model?

For us, it’s about learning and control. Running GPUs at scale was hard. 

The zero-to-one moment for LLMs is gone. Today, the innovation cycle in LLMs is mostly incremental—tool use, image generation, etc. The LLM world does not know where to go from here. Every lab is now looking at what comes after transformers because it’s unsustainable to scale just by adding compute. 

Everyone’s looking at alternative architectures. The SaaS space is being cannibalised by model service providers. These model providers can only make money if they're going to build enterprise apps. 

I see that coming in the next six months to one year. For an average SaaS company today, its biggest spend is on its cloud compute providers. Its second biggest spend is on its AI model service providers. Now, companies are giving tangible signals to these model service providers on where money is coming from.

A model service provider builds an application after taking 80% of your infrastructure spending, which isn't sustainable for the customer or for the company. That’s why we’re building in-house, so that our customer data remains within our ecosystem.

YS: What’s next in the roadmap for Zia LLM and Zoho’s AI efforts?

RR: It's non-negotiable that bigger models are better — we concur with that. A 7-billion model that we launch cannot do a lot of things, cannot be more generic, cannot have more emergent behaviour that it exhibits.

So our immediate goal is to scale this to 32, 70 and 100 billion parameters, and we are choked by hardware. That’s the reason we haven’t announced it yet. And we are also looking at other things like mixture-of-experts model training. So one is increasing the parameter size and trying out different strategies to make sure we stay on top of the benchmarks. 

YS: With Zia LLM, do you think India has had its Indic LLM moment?

RR: I would not say that. We’re not releasing it as a consumer-facing chatbot like DeepSeek did. At least for the next year, that’s not the plan. This is a business-focused LLM for Zoho’s suite. So yes, it's indigenous, but I wouldn’t go as far as calling it India’s DeepSeek moment.