Sarvam AI Releases India’s First Hindi LLM ‘OpenHathi’, a Week After Raising $41 million

Sarvam AI’s OpenHathi is the first Hindi LLM aimed at development of open models and datasets, to promote AI innovation for Indian languages.

Indian Generative AI startup Sarvam AI released OpenHathi-Hi-v0.1, the first Hindi large language model (LLM) in the OpenHathi series, just a week after raising $41 million in its Series A funding round led by Lightspeed Ventures.

Hindi is India’s most widely spoken language, with over 43% of the population declaring it as their mother tongue. The model is built on Meta AI’s Llama2-7B architecture and delivers performance on par with GPT-3.5 for Indic languages.

“We show that our model works as well as, if not better than GPT-3.5 on various Hindi tasks while maintaining its English performance,” the company said on social media platform X (formerly Twitter).

According to Sarvam AI, the goal of the OpenHathi series is to contribute to the development of open models and datasets to promote innovation in Indian language AI and is collaborating with academic partners at AI4Bharat, who have provided language resources and benchmarks for this initiative.

AI4Bharat is a research lab at the Indian Institute of Technology (IIT) Madras – a public technical university in India, which works on developing open-source datasets, tools, models and applications for Indian languages.

OpenHathi uses a 48,000-token extension of Llama2-7B’s tokenizer and undergoes a two-step training process. In the first phase, the model concentrates on embedding alignment, aligning randomly initialized Hindi embeddings. The second phase involves bilingual language modeling, where the model learns cross-lingual attention across tokens.

Full-Stack Generative AI Platform to be Launched Soo

Further, the company is encouraging people to be creative and come up with new ideas by using and improving upon this release. The company invites developers to build specialized models for various purposes using the OpenHathi-Hi-v0.1 model as a starting point.

Sarvam AI also plans to release advanced models for businesses on its comprehensive generative AI platform, which according to the startup will be launched soon.

While raising its Series A funding, the company announced its plan to develop the “full-stack” for Generative AI ranging from research-led innovations in training custom AI models to an enterprise-grade platform for authoring and deployment.

The company added that this full-stack approach will accelerate the adoption of generative AI in India, especially given that enterprises see the potential but are grappling with how to leverage it for their business.

Established in July 2023 by Vivek Raghavan and Pratyush Kumar, have previously worked at AI4Bharat – backed by Infosys co-founder Nandan Nilekani.

Recently, India launched BharatGPT, an LLM-based solution developed in collaboration with Indian conversational AI startup CoRover.ai, which is positioned as the answer to existing generative AI or LLMs. It intends to fill a critical gap in the market, providing an indigenous solution available in over 12 Indian languages, covering video, voice and text interactions.