The real deal

A Victoria University of Wellington researcher could hold the key to unlocking the commercial potential of ‘mixed reality’, a digital medium with a global market worth potentially billions of dollars. 

Dr Taehyun Rhee, a senior lecturer in Victoria’s School of Engineering and Computer Science, has developed a software solution that makes it easy and inexpensive to create mixed reality content (such as virtual objects)—requiring nothing more than a 360 degree camera, instead of the prohibitively expensive hardware and computational power currently required. 

So what is mixed reality and where does it fit in with its cousins—Augmented Reality (AR) and Virtual Reality (VR)? 

“Mixed reality lets the user see the real world (like AR) while also seeing believable virtual objects (like VR),” explains Dr Anne Barnett, Viclink’s General Manager of Commercialisation. “It then anchors those virtual objects to a point in real space, making it possible for the user to treat them as ‘real’. Imagine surgeons overlaying virtual ultrasound images on their patients while performing operations, for example.” 

Anne says that until now, the difficulties—and therefore the expense—involved in developing applications for the technology have centred around lighting and shading the virtual objects in a way that makes the synthetic content look as realistic as its real-world background, and how to make the real and not-so-real react to each other in real time. 

“With this software and a 360 degree camera, users can quickly and easily light or shade a virtual object,” says Taehyun (pictured, left). “The lighting and shading also move with the object, so that anyone can interact with it—for example, you could ‘virtually’ pick up a couch and move it all around the lounge to find the best spot for it!” 

With hugely positive feedback from the three public demonstrations of the technology in New Zealand so far, Anne says the commercial opportunities for Taehyun’s invention are extensive.

“From retail to education to manufacturing, mixed reality is positioned to drive business value across so many sectors—once the cost barrier is removed. We’re already talking with potential partners in California’s Silicon Valley and Japan.”

KiwiNet can also see the potential, awarding $250,000 of pre-seed funding to the project, which Viclink is matching dollar for dollar. The funding will be used in a number of ways—to support product development, engage with industry and end users, secure further investment and produce a short content demonstration for pitching at trade shows. 

Anne says they have already contracted additional expertise in the form of Dr Phoebe Kwan, Viclink’s Commercialisation Manager, Digital. Phoebe, who worked in the United States for over 20 years before moving to New Zealand recently with her Kiwi husband, has extensive experience and contacts in the emerging technologies of AR and VR. “She’s just the right person to help us take Taehyun’s technology to the next stage of development,” says Anne. “We’ve got some exciting times ahead of us.”