Homegrown startups developing LLMs that understand and respond in Indian languages

In November last year, OpenAIunveiled its AI-based service Chat Generative Pre-trained Transformer, popularly known as ChatGPT. The chatbot based on a large language model (LLM) caught the attention of businesses and people around the world who began using it for a variety of purposes. 

ChatGPT marked the tipping point for generative AI, which was followed by a series of other open models such as Bard, Mistral, and Llama 2. However, these models have limited or no support for Indian languages, making them unsuitable for non-English use cases.

Sensing an opportunity here, several homegrown LLMs have emerged on the horizon and are gaining significant momentum in the AI landscape. Apart from addressing language diversity, they also cater to specific cultural nuances. 

Interestingly, OpenAI CEO Sam Altman had earlier this year dismissed any significant competition to OpenAI from India. Now it may be too early to say if Indian models can pose a challenge to OpenAI, but they are making strong strides in the LLM race. 

Rise of Indian LLMs

Here’s a roundup of some homegrown startups in the LLM space and the unique models they are developing. 

Krutrim, Ola

Bengaluru-based Krutrim SI Designs, an AI startup founded by Olaco-founder Bhavish Aggarwal, has launched India’s own AI model developed from scratch. 

‘Krutrim’ (which means ‘artificial’ in Sanskrit) claims to be India’s first AI-based full-stack solution, built on Indian knowledge and data and Indian languages. 

Trained on 2 trillion tokens, this LLM comprehends over 22 Indian languages and responds/generates content in about 10 languages, including Hindi, Marathi, Kannada, Tamil, Gujarati, Odia, and Malayalam. 

Tokens serve as the fundamental components for processing and representing text data in LLMs.

The startup claims Krutrim was developed in just three months and is equipped with voice capabilities, demonstrates an understanding of the Indian ethos, and provides distinctive responses in various Indian languages. It also says Krutrim can surpass the capabilities of GPT-4 in Indic language support.

The company is also in the process of developing an advanced multi-modal model ‘Krutrim Pro’, which is equipped with problem-solving and task-execution capabilities. It is expected to be released in the next quarter.

Speaking at TechSparks 2023, YourStory's tech-startup event, in Bengaluru, Aggarwal highlighted that AI’s ubiquity could serve India only if it was created for the Indian context.

“Language is a challenge we face in India ... For a person who speaks English, access to opportunity has been easier. We need to build our own foundational models, our own libraries. India is a huge country and we have the advantage of data,” he said. 

Aggarwal recently announced on X (formerly Twitter) that Krutrim cloud will soon launch high-quality translation APIs for Indic languages, catering to text, speech, and video modalities, to build various applications and use cases across sectors.

OpenHathi, Sarvam AI

Bengaluru based Gen AI startup Sarvam AIhas released OpenHathi-Hi-v0.1, the first Hindi LLM in the OpenHathi series. By leveraging MetaAI's Llama2-7B architecture, the model delivers GPT-3.5-like performance, mainly for Indic languages. 

The company’s blog talks about difficulty with ‘tokenisation’ in models such as GPT-3.5. This means breaking down sentences in Indic languages can be challenging for computer models. 

OpenHathi is improving the way computers understand Hindi. It does this in two steps: first aligning Hindi words correctly and then teaching the model to understand words in different languages.

The startup focuses on training AI models for Indian languages and voice-first interfaces. It plans to collaborate with Indian enterprises for research-led innovations and seeks to combine generative AI with India Stack for public goods applications.

Sarvam AI has collaborated with multilingual AI chatbot KissanAI to enhance its base model using conversational data. This dataset includes conversations between a GPT-powered bot and farmers in different languages.

Project Vaani

Indian Institute of Sciences (IISc), ARTPARK (AI & Robotics Technology Park), and Google have collaborated on Project Vaani, which was revealed at the ‘Google for India 2022’ event. 

The initiative involves gathering anonymised speech data from 1 million individuals across 773 districts in India, emphasising diversity in linguistic, educational, urban-rural, and gender aspects across three phases.

Anticipating the creation of data corpora exceeding 1.50 lakh hours of speech, the collected and curated datasets will be open-sourced.

The project's primary goal is to advance automatic speech recognition, speech-to-speech translation, and natural language understanding. It aims to provide a technological solution that eliminates linguistic barriers and is accessible to a wider audience. 

BharatGPT, CoRover.ai

Bengaluru-based CoRover.ai offers BharatGPT, an LLM developed in collaboration with AI-based translation tool Bhashini under the National Language Translation Mission of the Ministry of Electronics and Information Technology. 

Utilising a multi-layered NLP (natural language processing) approach, BharatGPT employs various techniques, including natural language understanding, natural language generation, deep learning with generative AI, supervised learning, AI/ML, and context-based autosuggestion, to handle diverse user queries.

The startup offers support for building and managing chatbots across various communication channels and caters to institutions such as LIC, IGL, IRCTC, KSRTC, NPCI, and Government of India. 

It supports over 14 Indian languages through AI virtual assistants, including chatbots, voicebots, and videobots, for effective communication and interaction.

The company claims to have a user base exceeding 1 billion. 

Zoho

In June, software-as-a-service firm Zoho Corporationannounced its plans to build its own LLM, similar to OpenAI’s GPT and Google's PaLM 2.

Sridhar Vembu, Founder and CEO of Zoho, has said that the company is working on smaller AI models that are based on 7 billion to 20 billion parameters to solve the specific domain problems of its customers.

Indus Project, Tech Mahindra 

Six months ago, while in India, Altman sparked outrage when he said it was “hopeless” for a young team from India with limited resources to build a foundational AI model similar to OpenAI.

C P Gurnani, the then CEO and Managing Director of Indian IT major Tech Mahindra, responded on social media platform X, “Dear Sam Altman, From one CEO to another.. Challenge accepted.”

With the objective of laying the groundwork for Indian languages, Indus Project’s initial phase focuses on developing an LLM for Hindi, collecting 1.2 terabytes of data, across 40 dialects. 

Gurnani provided a recent update on X, “As I conclude my tenure at @tech_mahindra, I am delighted to announce that our ambitious #GenerativeAI project challenge has been successfully completed by our talented research team at Makers Lab.”

The beta testing phase for Indus Project has been launched. The LLM is trained over 539 million parameters with 10 billion Hindi dialect tokens, tweeted Gurnani. 

(The copy was updated with information on  Krutrim cloud infrastructure.)