San Francisco thought leader and star programmer Alex Kipman is already considered a legend at Microsoft in his early 40s. So it was less surprising that the native Brazilian opened the Microsoft developer conference “Ignite 2023” last year.
What was more surprising was how the developer appeared – as a hologram. A deceptively real illusion of his body stood on the floor of a virtual aquarium. Conference participants were able to beam into the world of fish with the Microsoft data glasses HoloLens 2. “We are entering a new era of collaborative computing,” proclaimed Kipman.
Microsoft’s latest project made it possible: “Mesh”. Mesh is a platform for “Mixed Reality”, also known as Mixed Reality or “Augmented Reality”. The real and digital world are connected with each other. A human can be present in both realities at the same time. “There are years of research and development behind it,” says Microsoft CEO Satya Nadella.
A key part of Mesh is the $3,500 Hololens 2 smart glasses, which can track hand, eye, and head movements and location changes. Objects in digital space can be touched and moved. The data glasses notice when the hand is stretched out and wants to grab something. It then transforms the real movement into the grip of a digital hand in real-time.
The glasses are connected to the cloud platform Azure, with its artificial intelligence and hundreds of possible apps. Read more at net2.com. According to Microsoft CEO Nadella, this creates a “global computer” where all the work is done. Corporate customers no longer have to worry about details, but only adapt virtual reality to their needs.
Microsoft provides developer tools to integrate existing software into Mesh. CEO Nadella wants to make common programs such as Office, Dynamics and of course Teams ready for the virtual world.
Teams should help Mesh to achieve a breakthrough
The Teams video conferencing service is Microsoft’s current bestseller , growing faster than Windows when it was first introduced in the 1980s. “Imagine a mesh-enhanced team where you can collaborate with colleagues from around the world as if they were all physically in the same room,” Kipman said.
Microsoft’s hopes are high. With Mesh, the company wants to increase the use and acceptance of its own cloud and catch up with the market leader Amazon AWS.
Every corporate customer can run their cloud software wherever they want, but mesh meetings take place on Microsoft servers. And the time for corporate customers is billed by the minute.
According to Forrester analyst JP Gowender, the “Internet of holograms could become the new standard and influence the future of work”.
According to surveys by the analysts of the consulting firm 650 Group from Incline Village in Nevada, all cloud companies with infrastructure for IT such as computing power, storage or network are already generating a turnover of 200,000 dollars per minute. Market leader Amazon alone rakes in 86,000 dollars in sales per minute and makes 26,000 dollars in profit per minute. Microsoft wants to go there too, and Mesh could help.
Google failed miserably because of the technology
Holograms are not new. Until now, however, they were more of a toy or useful for projecting extraterrestrial villains onto the bridge of the Starship Enterprise in Hollywood films. Mixed virtual worlds hardly played a role in daily use. First attempts like “Second Life” by Linden Labs suffered from technical problems, high costs and slow internet connections.
Google failed spectacularly with augmented reality. The $1,500 “Google Glass” glasses for the trendy computer nerd caused a stir in 2015 and disappeared again in 2019. But Google is not giving up either. In 2020, the internet giant took over the smart glasses manufacturer North. A start-up called Magic Leap is working on a new product for Google’s enterprise customers in the cloud.
“If you look at the history of IT,” explains John Hanke, head of games manufacturer Niantic from San Francisco, “it has always moved from business life to private life. Never vice versa. That’s why Google Glass failed. Consumers wanted them first. But Microsoft is big in business.”
Niantic was spun off from Google in 2015. According to Hanke, it now has 600 employees and has annual sales of over a billion dollars. The main attraction is “Pokemon Go“, an augmented reality game in which you have to conquer enemy target points with your Pokemon, friendly characters with magical abilities.
The rest then runs digitally on the smartphone. On stage, Hanke showed what it would look like to walk the streets wearing smart glasses, your Pokemon by your side, then encounter another “Trainer” with your Pokemon and be able to challenge them to compete.
Hanke thinks a lot about the competitor Mesh. “Microsoft is a pioneer here. When they come from the business to the consumer market, we want to be ready.” He has no doubts that this will happen. “The PC wasn’t the end of evolution, the laptop wasn’t, the tablet wasn’t, and smartphones won’t be either.”
Niantic itself is developing the prototype of data glasses together with the chip giant Qualcomm .
Grab the Excel spreadsheet with your hands
Mesh solves many problems in times of pandemics. At a time when people are less able to travel, work or party in groups, technology offers a way to at least get back together digitally.
Anything that can be digitized can be grasped or modified. Design models or Excel data. The pillars of a spreadsheet no longer exist only on the screen, they pop out of the floor in the conference room. The participants in a meeting walk past them, can implement changes and experience the effect. “In a hybrid world of work,” says Kipman, “’here’ can be anywhere. Mesh will allow organizations to hold meetings where everyone is in the same room.”
“That’s the future. The only question is when it will be ready,” says Holger Muller from Constellation Research in California. The current situation is unsatisfactory. “Humans,” he says, “are three-dimensional computers. So far, his digital experiences have always been a compromise. It is only through the human ability to abstract that we can use devices and screens to connect real and digital worlds.”
Industry 4.0 applications
Analyst Muller is building a bridge from the “Internet of holograms” to the “Internet of Things”, where billions of sensors will soon digitize and connect factories and corporations down to the smallest switch via the cloud. “I then no longer pull out a laptop or smartphone, but go into the factory building with the HoloLens, and all the machines talk to me, signal problems or warn of failures,” he says.
“I become part of the factory and bring a technician avatar into the hall at any time, regardless of where the actual technician is stationed,” Muller continues. “We then look at the machine, not at a laptop.” The consulting firm Accenture is planning virtual office floors for companies with “nth Floor” so that employees can work there together.
House plans on paper are also obsolete. Life-size digital models will be digitally accessible on the construction site, and change requests can be assessed in a matter of seconds. However, two industries should not be happy if Mesh can prevail and a HoloLens is as big as a pair of sunglasses and costs $200, believes Muller: “Then I don’t want to be a monitor or smartphone manufacturer. Because building a phone or SMS function into a HoloLens would not be a problem.”
Microsoft currently has two applications available. The mesh platform is in beta for Hololens 2 and a mesh-enabled AltspaceVR app for use with laptops, smartphones or virtual glasses such as those from Oculus , HP or HTC .
Apple and Facebook are also looking into the technology
Apple is also working on the technology. Rumors have been growing since February about augmented reality glasses called “Apple Glass”, which could be released in 2023. It would be a consistent continuation of CEO Tim Cook’s strategy. With the Watch, he has already seized the market for “wearables”, computers that can be worn on the body. Apple Glass would be the ultimate wearable supercomputing device.
Speculations assume that the glasses will not be as powerful as the HoloLens and that they will use the processing power of an iPhone. This should keep power consumption, weight and costs low.
Facebook’s Mark Zuckerberg also announced smart glasses and a collaboration with eyewear manufacturer RayBan. Products are expected between 2023 and 2024.
There’s no question that it will be some time, says analyst Anshel Sag, before a “mixed reality” becomes as normal for us as a phone call. “But mesh makes that much more likely,” said Sag, who works for industry consultancy Moor Insight & Strategy.