Hinglish, Bengalish, Tamlish: How Kolkata-based Mihup is adding a multilingual twist to in-car voice interactions

After Alexa and Google Echo found their way into homes, vehicles have been the next best use case for AI-based voice assistance to proliferate. However, despite being around for some time now, in-car voice assistance solutions come with a limitation - the predominance of a single language (US English), narrowing the reach of technology, especially in a multilingual country like India.  

According to a survey, non-American users find it 30 percent harder than Americans to understand existing western origin voice assistants in cars. The limited dictionary and regional languages support has been largely absent in the software provided by companies in the past.

Kolkata-based startup Mihup is attempting to overcome this language limitation with its AI-powered vernacular voice interface for vehicles. The startup has developed a software—AVA Auto—for automobiles that allows users to interact with a car system in multiple languages. 

In a unique offering, the system supports various Indian dialects, including mix slangs like ‘Hinglish’ (Hindi and English) and aims to introduce AVA support for ‘Tamilish’ (Tamil and English) and ‘Benglish’ (Bengali and English) soon, with a plan to cover all major Indian languages by 2022.

The offline model (Voice AI on the edge) of the product has been integrated with the infotainment system of two Tata Motors cars—Tata Altroz and Tata Nexon, by setting up the solution in around 50,000 Tata cars and other commercial vehicles and is soon planning to enter two-wheelers. The startup has been registering 20 percent month-on-month growth and aiming to penetrate its vernacular voice technology across the automobile sector in India. 

Single tech, multiple-use case 

Mihup is the brainchild of Tapan Barman and Biplab Chakraborty. Incorporated in 2016, the Kolkata-based platform had started out with conversational AI solutions and voice assistance for primarily call centres. It recently entered into the automobile sector, which now holds 30 percent share in the overall business. 

“The call centre is a huge market and bigger than auto. The revenue share from this segment will continue to remain high going forward. The auto segment is just an example of what can be done with voice technology across segments,” says Co-founder and CEO, Tapan.  

Moving forward, Mihup aims to give the platform to developers who will be able to create a voice experience of their choice across applications. Being horizontal in nature, the platform and the voice technology is sector agnostic and can be deployed across categories like consumer durables and wearables. 

So, while the platform can work for multiple applications, the startup has laid its focus on the automobile sector, where the demand and use case for voice assistance is high.

ALSO READ

Conversational AI startup Uniphore leverages AI to ready for post-coronavirus world

The auto platform

Talking about Mihup’s AVA auto platform, it offers online (using cloud or internet connectivity), offline (without internet), and hybrid connectivity options among other features. The system responds back based on the user’s voice command without the need for a manual switch. This is one of the unique offerings of the software, says Tapan. 

Besides these, the technology also enables the driver to access hands-free voice-based control car functions like air-conditioning, AM/FM, volume, and music system. It provides natural language support for making phone calls and playing music from USB, supporting navigation commands and also for vehicle information.

Mihup has recently replaced the software offered by American technology company Nuance, which was being used by Tata Motors since 2014 for their smart assistant in vehicles. While the in-car voice assistance provided by Nuance was proficient, it came with a limited dictionary coverage and regional languages support, an area where Mihup is building the first-mover’s advantage. 

“It was an incumbent solution with a limited command set and it was not providing a natural experience to the users. We changed the entire part, brought in natural language support, which helps users control everything in the car by talking naturally. Dialect recognition and accuracy is our major USP,” says Tapan. 

Competitive edge 

Mihup’s technology - the ability to understand mixed-language support - has been patented by the startup. 

Amazon, Google and Apple are the biggest names in voice technology with their products Alexa Auto (Alexa Custom Assistant, Echo Auto), Android Auto and Carplay, respectively. The companies have integrated their voice technology in vehicles by collaborating with major automakers like Toyota, BMW and Ford, Hyundai, Honda, and BMW. We are likely to see the system in some of the future models in India as well.

Technology leader IBM has also launched its white label solution—Watson Assistant.

On the other hand, OEMs are also developing their own voice assistants by turning to natural language processing startups and tech companies. Some of the popular names in this segment are US-based companies Cerence, SoundHound and Nuance Communications, which have all collaborated with global automakers like Honda, Hyundai Motor Group, Kia Motors and Daimler, Toyota and so on. 

“Vernacular as a support is a global issue and not restricted to India. But the way we have developed the solution for the Indian context is unique and gives us an edge over other companies,” says Tapan. 

There is scope for multiple virtual agents to co-exist with a single resident voice engine at the helm. Every OEM wants to create their own experience for the consumer and protect their brand identity and privacy and hence can choose to integrate any third-party voice engine such as Google assistant or Siri on the top of the platform.

“It will be a multi virtual personal assistants (VPAs) approach. Users will be able to choose either of the virtual agents like Alexa or Siri, but OEMs will have some resident voice engine (like Mihup), which will arbitrate in the system and will have the control,” says Tapan.

ALSO READ

'Voice assistants, augmented reality will transform retail'

Going beyond infotainment

Until now, we have been using voice commands for features such as playing music, making calls or navigation, while travelling. However, the embedded voice AI will offer a lot more diversity of features and benefits for users, car makers and businesses in the entire ecosystem. Vehicles will soon become new customer touch-points for a plethora of businesses, says Tapan. 

“In the next 24 months, our system will support multiple things which will be way beyond what we offer currently,” he adds.

First, users will be able to sync their home devices like Alexa with the car system and hence be able to control everything from the car itself.  

Second is the development of vehicle commerce where passengers will be able to book or shop through in-car voice assistance and command. For instance, the system will inform passengers about certain points of interest while travelling like nearby shopping discounts, events, fun cafes, historical places and so on, as they pass through those streets. Advertisements and content consumption in cars will benefit telecom companies, cloud service providers, OTTs and ecommerce companies. The integration will be based on open-source APIs. 

The third feature is ‘speaker verification’, which will help authenticate the identity of the driver or the co-passenger (virtual biometrics). This will be beneficial in cabs and public vehicles.

Funding and market 

From vehicle manufacturers to technology giants, pretty much everyone is warming up to the voice-enabled ecosystem, and as per International Data Corporation (IDC) estimate, there will be 76 million connected cars shipped globally by 2023. 

“It’s a $300 billion market globally for auto solutions. The virtual agent in the car is going beyond normal controls and unlocking areas like vehicle commerce. A whole new ecosystem is getting created inside the car,” says the founder.  

Backed by Accel Partners and Ideaspring Capital, the startup has raised a total of $10 million in funding over three rounds. Their latest funding was raised in December 2020 with a Series A round of $1.5 million, with participation from Rajesh Jain, Founder of Netcore, and Jayant Kadambi, Founder and CEO of YuMe Networks. 

Mihup was recently selected by MG Motors under the MG Developer Program & Grant 2.0, where the OEM major will explore synergies in voice technology. 

“They (MG Motors) were excited to see a solution like ours being developed in India. We are in discussion to work with two to three major OEMS in India and abroad and targeting a revenue of $5 million in next 12 months,” he says. 

The startup has onboarded US and UK English and Portuguese language to target OEMs in multiple markets like North America, UK and Brazil. In a first, it will be deploying the solution in commercial trucks by the end of 2021 and looking at two-wheelers as well. 

YourStory’s flagship startup-tech and leadership conference will return virtually for its 13th edition on October 25-30, 2021. Sign up for updates on TechSparks or to express your interest in partnerships and speaker opportunities here.

For more on TechSparks 2021, click here.