Building foundational AI in India: The gap between ambition and infrastructure

The gap between having GPUs and actually getting them to work is wider than most people think. That's the reality check India's AI builders are dealing with right now.

At a recent mixer in Delhi, E2E Networks and YourStory brought together voices from different parts of the AI stack to talk about what it actually takes to build foundational models in India.

E2E's infrastructure plays

Chirag Anand, Senior VP of Product Development at E2E, walked through E2E's cloud GPU platform. The company operates entirely in India, with data centers in NCR, Mumbai, and Chennai. It's NSE-listed and holds ISO 27001, SOC 2, and PCI DSS certifications. The platform supports everything from traditional CPU workloads to GPU-specific services for AI training and inference.

The demo covered the managed training cluster service, which simplifies building clusters for model training. Users can select from H200s and H100s, scale up to 1,000 GPUs, and get InfiniBand interconnect running at 400 Gbps. The platform also offers managed inference services for hosting open source models or custom models, with auto scaling and load balancing built in.

Anand also mentioned fractional GPUs, which let multiple smaller models share the same GPU while staying isolated from each other. If one model crashes, it doesn't affect the others. The use case is straightforward: hosting multiple small language models on the same GPU to maximize resource utilization.

Scaling AI video generation

Devender Bindal, Co-founder and CTO of TrueFan, showed a demo of what TrueFan has built with AI-generated video content. The company trains models on celebrity or brand ambassador content, then uses those models to create personalized and localized videos. The company has scaled across 175 languages.

The demo showcased GoIbibo and Shri Ram Finance. For GoIbibo, TrueFan produced over 1.5 million personalized videos for users who abandoned bookings, tailored with names and available in multiple languages. For Shri Ram Finance, it created various localized ads using limited celebrity content featuring Rahul Dravid.

Infrastructure as competitive advantage

The panel discussion, moderated by Shivani Muthanna from YourStory, dug into what it means to build foundational AI models in India. The topic was ‘India's AI Mission in Action: Building Foundational Models on E2E Cloud Infrastructure’.

When asked about the biggest gaps startups face, Anand pointed to the newness of the technology. "Founders are actually struggling to get things up and running," he said. "The gap between getting the GPUs and actually getting them to work is something which I've personally seen taking some time. It has nothing to say about the competency of the founders or the developers. It's just because things are so new."

Bindal shared a concrete example of scaling challenges. On Mother's Day 2024, TrueFan helped Zomato generate 5 lakh videos in a day. At peak capacity, they were producing 2,500 videos per minute. "The main issue was procuring the number of machines that we wanted to do, and also the cost associated with those machines," he said. "You cannot have machines idle, as they are expensive. Indian founders have to make sure they can scale with certain costs as well."

That cost pressure forced architectural decisions. TrueFan had to rearchitect its lip sync model to run on smaller machines like Nvidia T4s and L4s instead of H100s. Not because the bigger machines didn't perform better, but because the capacity needed wasn't available in India at scale, and the economics wouldn't work. The shift reduced costs 50-60x while maintaining quality.

Ashish Taneja, Founding Partner and CEO of growX ventures, brought an investor's perspective on what's changed in India's AI ecosystem. "Earlier, a lot of founders were building wrapper solutions," he said. "The first shift we've seen over the last few months is that local companies are deploying strong AI solutions. Be it financial services, defense, healthcare, or multiple other use cases."

He emphasized that India's market dynamics differ from Silicon Valley. "India as a market is very, very cost-conscious. Indians tend to wait for the cost cycles to reach a level where it is affordable, where it's manageable, and then deep dive," Taneja noted.

What the India AI mission enables

The conversation touched on the India AI mission's 177 crore allocation and how it's changing what startups can attempt.

Anand explained that beyond GPU availability, the mission has made GPUs more affordable. "The use cases that would require maybe larger capital have been enabled just because they are available cheaply," he said.

When it came to infrastructure requirements, Bindal was direct about non-negotiables. "The availability of the machine should be 99.99% available. Security also has to be ensured. The platform should be compliant. And the performance of the machine is very important," he said. TrueFan runs benchmarks across different providers and machine types, and when it finds performance differences, it works with teams to solve the issues.

Taneja explained how infrastructure choices factor into investment decisions. "GPU-oriented questions are asked for AI people, because that's the core," he said. "If you don't have that capacity, you're out of the game." The due diligence covers access to high-quality GPUs on demand, whether capacity is future-proofed, and whether the cost curve makes sense.

But he also pushed back on the idea that Indian startups should chase the same problems as US companies. "We should not get excited about building these large language models," Taneja said. "Build local where we have the data, which is proprietary and defensible."

Practical choices

Anand highlighted two patterns he sees with generative AI customers: experimentation and scalability. With reserved capacity, teams don't have to hunt for GPUs. They can run multiple experiments and move to market faster. When they need to scale, having capacity available means infrastructure isn't the bottleneck.

He also emphasized the advantages of local infrastructure. "If most of your customers are in India, host your servers in India," he said. The latency and cost effectiveness aren't minor details; they affect whether the economics actually work.

His closing advice was practical. Choose a stack where you can move fast. Pick a provider you're comfortable with who offers the services you need. Keep customer needs in mind when making infrastructure decisions.

The mixer didn't offer neat solutions to India's AI infrastructure challenges. But it showed the gap between national ambition and ground-level execution is closing. Teams are figuring out how to train models on T4s instead of H100s. They're generating millions of videos without burning cash on idle machines. They’re choosing local infrastructure because latency and cost actually matter. And they’re building AI that is defensible not because it is bigger, but because it is relevant to India’s constraints and customers.

It may not look like a moonshot moment. But it is progress measured in uptime, cost curves, and working deployments.

And increasingly, that’s where the real work is happening.