This student innovator's tech makes VR more social by doing away with headsets

Twenty-six-year-old Arvind Sanjeev's Lumen allows people to feel immersed in Virtual Reality (VR) in their natural space without using headsets.

For all the hype around Virtual Reality (VR) and Augmented Reality (AR) technology (the market is estimated to grow at an impressive CAGR of 57 percent in 2018-2022), there seems to be one snag. By its very nature of requiring headsets or multi-projected environments, it socially isolates users.

People are unable to share the experiences they have while using these headsets with others who are not wearing them, thus “socially handicapping” people.

Research shows that experiences generated using such technologies in the long term dissociate people from reality. While the constraining movement to a particular location through the tether on the headset only makes it worse.

“Technology was meant to be designed in a way that it adapts to people, and not in a way where people have to adapt to use such technologies. Do we really want to live in a future where everyone has a headset on their face?” asks 26-year-old Arvind Sanjeev, who has come up with a solution where people can feel immersed in VR in their natural space without using headsets.

Named Lumen, the technology is developed by Arvind at the Copenhagen Institute of Interaction Design. Lumen is a Mixed Reality storytelling platform that challenges this trend of social isolation. Giving examples of its use cases, Arvind says this technology can be used in museums or even direct the user on the road to their destination.

“The main inspiration behind Lumen was imagining a future where people’s imagination is no longer confined to 2D touch screens or headsets, but instead by projecting it outside into the environment this would allow them to interact with the world around them,” he tells me in a chat over WhatsApp from Copenhagen.

Design thinking

Lumen uses Machine Learning (ML) and projection mapping to explore the creation of a new kind of media that takes advantage of the physical world while overlaying a layer of digital fiction on top of it. The project started off with the design question: "How might we immerse people in alternate reality experiences without isolating them in headsets?"

Though it started out as his Master's project, Arvind is talking to various companies, including Apple and Lego, to see how he can collaborate with them.

Before he joined the Copenhagen Institute of Interaction Design, Arvind had graduated as an engineer from the TocH Institute of Science and Technology in Cochin. During his undergrad days, he founded a startup, DIY hacking, which is now called Maker.pro, and collaborated on another one called Fin Robotics. He is not associated with them anymore, but would now want to take his new VR project to market.

Arvind says Lumen was created through a people-centred design process by talking to, co-creating, and collaborating with different people. “Moreover, co-creation and prototyping sessions with a LARP (Live Action Role Playing) story master, Troels Andersen, were really helpful for understanding the elements and experiences that create immersive stories. As part of the research, escape rooms and museums were also studied for understanding the immersion they are capable of creating,” he says.

For exploring the interactions, several prototypes were made such as for storytelling with kids, storytelling in art galleries, storytelling in homes, etc. And after testing out each of these prototypes with people, it was understood that the potential for this kind of tool lies in it being a platform, a storytelling platform that lets people tell their stories through magical interactions with their environment.

Stating that the inspiration for the design came from our deep connection with flashlights, which we have all played with as kids and used them to tell stories, Arvind says Lumen borrows this metaphor to help people create a strong mental model around using it. “You can use it just like a flashlight, by pointing and clicking on anything to reveal something hidden,” he adds.

He is working on making the design sleeker or even adapted to smartphone cases to make it more ergonomic.

Technology behind Lumen

Lumen uses the yolo darknet machine learning platform to classify objects, which is then processed by the onboard algorithm that generates stories on top of those classified objects. Narratives for this platform are designed by storytellers and game designers who will have access to the background graphical interface that serves as the story builder and dashboard.

“The hardware consists of a laser projector in combination with a camera and depth sensors that work to create the best projection mapping experience for the user,” he says.

An Interaction Designer, Arvind hope to design a people-centred future through technology. Through Lumen, he wants to provoke the tech industry, which is burgeoning with augmented reality and virtual reality-based technologies like the Microsoft Holo lens, Oculus Rift, HTC Vive and many more currently focussing on only headsets to deliver immersive experiences, to think outside them and design a future where people are not constrained by technology.