From planning to production: How TrueFoundry simplifies machine learning deployment
TrueFoundry helps users turn ML models built on tools such as Jupyter Notebooks into live APIs, making it easier to build, track, and deploy models without deep knowledge of systems like Kubernetes.
Building a machine learning model is only half the journey. Getting it to run reliably in the real world often requires navigating complex infrastructure. The transition from building on Jupyter Notebooks to creating a production-grade service typically demands DevOps, which can be quite challenging for many.
This is where TrueFoundry comes in. It offers a platform that runs on Kubernetes, but shields users from its complexity, helping enterprise AI/ML teams build, deploy, and manage large language model applications easily.
Kubernetes is an open-source system that helps manage and orchestrate containerised applications. It helps run and manage these apps so that they stay up, scale when needed, and recover if something goes wrong.
The San Francisco-based enterprise platform-as-a-service (PaaS) was founded by Anuraag Gutgutia, Abhishek Choudhary, and Nikunj Bajaj in 2021.
The company sees most traction from the US, and also some from Europe and India. It sees traction from healthcare segments including manufacturing, pharma, insurance, and medical devices; software and SaaS; and financial services.
The beginning
TrueFoundry traces its roots to a bottleneck the founders faced with their previous venture—Enthire, which was later acquired by InfoEdge. While trying to integrate their ML model to a web app, they realised that it takes a longer time and requires deep understanding of cloud, infrastructure, compute, and Ops.
Prior to starting their own venture, Choudhary and Bajaj both worked at Meta, where Choudhary was leading the videos team at Facebook as a senior staff software engineer; and Bajaj was part of the machine learning team.
“Meta has an internal platform that simplifies the life of data scientists where they don't have to ever worry about infrastructure and compute. As long as they write the business logic, the platform takes care of everything and ensures that data scientists don't make mistakes,” Anuraag Gutgutia, Co-founder and COO, TrueFoundry, tells YourStory.
According to him, Meta’s platform reduces the data scientist's dependency on platform teams by acting as the team itself. This allows data scientists to not just experiment, but push the models or applications that they are building to production and see it live.
The founders wished to create something similar and provide it to other companies globally, so that they can “care about the creative part and everything that is non-creative is abstracted out from them.”
They also noted that in the context of an enterprise, the systems would have to be multi-cloud, and that in the future, people might have on-premises and cloud operations, which lead to their next conundrum—how to build architecture that allows future scalability and flexibility to have newer types of models, applications, and hardware easily supported by the system.
This was made simpler with the entrance of GenAI, which also marked an increase in GPU usage. Gutgutia adds that the architecture designed by the company initially allowed them to scale quickly in their product journey from a GenAI perspective, allowing them to tap into a number of enterprise clients.
Products and offerings
The company broadly classifies its suite of product offerings into four categories.
First, Model Serving, is a set of tools that caters towards the deployment of any AI model or application, where data scientists can take their entire process from experimentation to the final stage of deployment to scalably running it on cloud or on premises using TrueFoundry’s system, explains the COO.
The second, LLMOps, revolves around LLMs and Gen AI, and is specific tooling for GenAI that enables enterprises to build and ship compound AI applications. This involves LLM deployment, fine tuning and the ability to package compound applications and then deploy it into TrueFoundry’s system, he adds.
The third, AI gateway, acts as a layer to enable enterprises to have a common view into all their GenAI processes. They can access it from the same endpoint and have some level of governance around it in terms of access control, cost management, rate limiting and so on, says Gutgutia.
Lastly, there is RAG, a template that allows enterprises to take the open source framework, use it to build their own internal templates and deploy it into TrueFoundry’s system and expose it as a reusable framework for other teams internally, he adds.
According to the founder, the first two categories overlap in terms of architecture and design, and are similar. The third—the gateway—is a specific part of the second, and they are generally bought together.
The B2B company follows two models for its pricing. For startups and mid-market companies, it is priced on a per-developer basis. Meanwhile, for enterprises, where the entire system can be deployed internally to an enterprise Virtual Private Cloud (VPC), it is still priced based on the number of users, but has a minimum of 15 users. Thus, the pricing for enterprises ranges between $100,000 and $250,000 depending on the set of features they use. As enterprises scale and the number of developers expands, the price per developer reduces, says the COO.
Funding and future
In February, TrueFoundry raised $19 million in Series A funding led by Intel Capital. The round also saw participation from Eniac Ventures and Peak XV’s Surge, Jump Capital, Gokul Rajaram, Mohit Aron, and Cyan Banister, among others.
Going forward, TrueFoundry plans to focus and double down on the US markets where it sees a huge potential with “more than 3,000 companies with more than 250 million in revenues,” says Gutgutia.
The company also plans to continue to look at other markets for expansion.
With a team about 50 strong, it works with over 25 Fortune 500 companies and midmarket companies including WadhwaniAI, WhatFix, Games24x7, AvisoAI, Siemens, Alineo and Nvidia.
According to the co-founder, TrueFoundry competes with the internal platform teams of enterprises, and the platforms of individual cloud providers.
While the company did not disclose its revenue figures, it said that it projects to grow 3x this year. In the next five years, the company aims to become the de-facto platform for anyone doing anything on AI.
While it currently allows models and applications to be deployed on its system and enables users to run their compute efficiently, TrueFoundry aims to turn its platform into an autopilot system which can take care of everything once the code is added.
“So far India has built a lot of consumer companies and we have seen amazing companies come out of India. On the software side also, we have seen some super good tools like Postman and all come out. We haven't really seen a super big company that has come out in the data or the AI space from India that is playing at the level of infrastructure. And we believe that is a huge opportunity, and hopefully we are able to be that company that puts India on the roadmap for AI infrastructure,” Gutgutia concludes.
Edited by Jyoti Narayan