Microsoft and Volkswagen make HoloLens 2 usable in moving vehicles

Microsoft and Volkswagen have collaborated on algorithms that should make the HoloLens 2 more usable in continuously moving vehicles. This was previously not possible because the AR headset’s sensors got confused in moving vehicles.

From a Microsoft blog post It turns out that in 2015, the German car manufacturer was already investigating how augmented reality could be used in car traffic. Volkswagen used, among other things, the HoloLens AR glasses from Microsoft, but ran into problems during the tests with those glasses. “When the AR glasses were placed in a moving vehicle, the sensors were no longer able to track the environment and the holograms in the screen disappeared,” said an employee of the company.

Volkswagen shared the issues with Microsoft’s team developing the Hololens, and the two companies decided in 2018 to investigate the issue and come up with a solution. In this way, the researchers found that cameras in the AR headset could not detect movement in a moving vehicle, while the internal motion sensors could.

This created a discrepancy between the data that both sensors were able to collect, as a result of which the glasses no longer functioned properly. An algorithm was then developed that had to resolve the difference in measurement results from both sensors and that algorithm was subsequently extensively tested and found to be good.

Microsoft briefed companies in early 2022 about the progress it had made. According to the American company, some maritime organizations have already shown interest in using the Hololens 2, thanks to the new technology, in their maritime working environment, which is also often far away. Currently, the feature works on large ships, according to Microsoft, but the company says it is busy making the platform also compatible for use in moving elevators, trains, cars and other moving environments.

HoloLens 2 holograms