NeuralGarage seeks to remove audio-visual dissonance in dubbed content using generative AI

Note: NeuralGarage was featured in YourStory in July. This is a truncated version of the profile, with updated information on the startup’s entry into YourStory’s Tech30 list, as one of the 30 most promising Indian startups of 2023.

Today content is dubbed in several languages to reach a wider audience. However, the dubbed content doesn’t always provide a cohesive viewing experience as the lip and jaw movements of the actors do not synchronise with the words coming out of their mouth.  

NeuralGarage aims to address this issue with its flagship AI-powered product VisualDub—so that viewers have an immersive experience and are able to appreciate the visual aspects fully.  

“I have a great fondness for Korean films, and this was clearly at its peak during the lockdown. But I had problems with the visual dissonance due to the lack of audio-video synchronisation. This sparked the idea of whether this could be solved,” says Anjan Banerjee, one of the founders of NeuralGarage. 

Driven by his desire to bridge the gap between audio and video, Banerjee began studying the potential of AI along with his batchmates from IIT Kanpur, Subhabrata Debnath and Subhashish Saha. The three of them had earlier co-founded Visage Map, a facial recognition startup, which was later acquired by FaceFirst, a US-based facial tech company. 

The trio reached out to Mandar Natekar, a media and entertainment veteran, for mentorship, which ultimately paved the way for the birth of the deeptech startup NeuralGarage in 2021. 

“Our vision at NeuralGarage is to make communication seamless across all barriers of language visually through the power of AI,” says Natekar, Co-founder and CEO, NeuralGarage.

The name ‘NeuralGarage’ stands for neural networks—which is the core of AI—and pays tribute to tech companies Apple, Microsoft, Google, and Meta, which were supposedly started from a ‘garage’.

Eliminating audio-visual discord

NeuralGarage’s flagship product VisualDub reduces audio-visual disparity in dubbed content by syncing the lip and jaw movements of actors with the audio—using proprietary generative AI (GenAI) technology.  

GenAI transforms facial parts using audio activations, blending them with the rest of the scene. The lip movements are tweaked to match syllables, and the jaw and chin movements and smile lines are harmonised with this to make the dubbed content visually realistic and natural. This technology is person- and language-agnostic. 

“Removing visual dissonance makes the dubbed content look authentic and local, which helps viewers and consumers connect more with the content,” says Natekar. 

NeuralGarage offers this technology through API integration, SaaS, and desktop software. 

The technology has been tested in more than 30 languages across the world, including many Indian languages and international languages such as Italian, German, Spanish, Japanese, Korean, and Mandarin. 

Business and growth

NeuralGarage generates business from advertising, influencer marketing, content creation, OTT, and films. Film and edtech content account for more than 90% of its revenues. All these projects are under process. 

The startup has 10 clients including Amazon India, Microsoft, Hippo Video, and Pixis. 

Recently, Amazon’s ad campaign with actor Manoj Bajpayee was shot in Hindi and dubbed in Tamil, Telugu, Kannada, Malayalam, Bengali, Gujarati, and Marathi. 

VisualDub was then used to lip-sync the creative in the dubbed languages to give the feeling that the creative was actually shot in these many languages, thus creating an authentic connection with the consumer, explains Natekar.  

NeuralGarage—which is backed by institutional VC fund Exfinity Venture Partners and angel investors including Amit Patni—has raised $1.45 million to date in its seed round.

Also Read

Nawgati: An AI-powered platform that provides real-time updates on congestion and wait time at fuel stations

There are over 60 startups in the generative AI landscape in India, across various industries, according to NASSCOM. They include TrueFoundry, which helps build ChatGPT with proprietary data; Cube, which enables AI-based customer support on social media, and Gan.ai, a generative AI video platform. 

This year, NeuralGarage aims to drive commercial testing and adoption of VisualDub across verticals and explore different use cases. The startup is targeting a revenue of $1 million and 50 clients by the end of this year. 

Participation at TechSparks

NeuralGarage was part of TechSparks 2023, YourStory’s flagship tech startup event held recently in Bengaluru, and made it to YourStory’s Tech30 list as one of the 30 most promising Indian startups of 2023.

“YourStory’s flagship event TechSparks is a valuable resource for early-stage startups. The platform's ‘Tech30’ event brings together top startup founders, providing valuable insights and best practices,” says Natekar. 

“The team responsible for the pitches was excellent, providing the necessary support to present impactful stories. Alakh Pandey, Nitin Kamath, and Bhavish Aggarwal shared candid stories of their journeys and consumer success, which are valuable for early-stage startups,” he adds.