Could mixed reality overcome the language barrier?

For centuries technology has been gradually bringing people from different countries and cultures closer together in various ways. But even as it has enabled us to communicate quicker and easier than ever before wherever we are in the world, language barriers have remained tough to crack.

Nevertheless, language service providers, or LSPs, have been in fierce competition to try and come up with the perfect solution for cross-language communication. It’s a rapidly growing market – in the past ten years, the translation industry has doubled in size, from being worth $23.5 billion in 2009 to around $46.5 billion today. This has led to major leaps forward in the technology available for those wanting to try out a new tongue, but whatever you opt for, there’s nothing out there that can have you actually speaking another language yourself – not without putting you through hundreds of hours of tuition anyway.

Or is there in fact a fast-track way to become multilingual? Let’s take a more specific example. What if you were a respected professional doing a talk at a conference that’s being streamed online, but one of your target audiences is in a place where you don’t speak the local lingo? Not an uncommon situation these days.

From the audience’s perspective, watching someone delivering a speech in a language that is not our own from the other side of the world and being able to understand what is being said is already possible through an internet connection and a translator, but it isn’t exactly the most engaging of experiences. Many would agree that being able to see and hear the person on stage, in their own voice but most importantly in your language would make it far more absorbing. But if they don’t speak your language there’s nothing that can be done, right?

HoloLens 2: the next big translation tool?

If Microsoft’s recent stunt involving its new HoloLens 2 is anything to go by, mixed reality could turn out to be a highly effective method of using technology to communicate with someone that speaks another language.

At the company’s Inspire event in Las Vegas earlier this year, Microsoft’s senior vice president of Azure marketing, Julia White, donned one of the firm’s latest headsets and appeared to conjure up an AR clone of herself live on stage. This might sound impressive already, but there was more – it started speaking Japanese, and White can’t speak Japanese.

But how is that possible?

To create the ‘hologram’ a volumetric capture of her body was carried out at a mixed reality studio, while the speech element was made possible using Microsoft’s AI neural TTS (text to speech) service, which picks up the unique characteristics of a person’s vocals to create a “personalised voice signature”. Once the program had learnt how to speak like her, it could use the new preset to read out a piece of text in Japanese and hey, the trick was complete.

Of course, this isn’t the same as speaking to someone from a different country and having your voice immediately translated into that person’s language in real-time. You still have to ‘feed’ it the text manually.

Then there’s the fact that it could be some time before HoloLens 2 headsets are readily accessible for people wanting to make this kind of interaction. Although it was officially unveiled to the world at Mobile World Congress in February, it’s still not entirely clear when the units will start shipping, and the $3,500 price tag may not be suited to all budgets. But even if it could be a while before customers are able to get their hands on this kind of technology, it’s an interesting sign of where things could be going in the field of technology for communications.

Whether holographic communication via avatars is on the horizon or not, it does appear that out of all the tech giants currently jostling for a leading position in the AR/VR/MR space, Microsoft is a few steps ahead in this particular category. It has been more than three years since the manufacturer first went public with the work it had been doing in the 3D communications space.

Back in 2016, around the time that the original HoloLens launched, Microsoft unveiled ‘Holoportation’. It described this as a new type of 3D capture technology that allows high-quality three-dimensional models of people to be reconstructed, compressed and transmitted anywhere in the world in real time. This, the company says, makes communicating and interacting with remote users as natural as face-to-face communication.

That may be open to debate, as many will argue that despite the myriad of ways that we can now exchange information over long distances, nothing can really match a real in-person encounter. But when we’ve got companies taking technology that previously could only be seen in Star Wars and making it a reality, who can really say how we’ll be communicating in the not too distant future?

2019-08-29T17:35:15+00:00 August 29th, 2019|VR News|0 Comments

Leave A Comment