The MR360 toolkit automatically analyses 360° video and detects lighting conditions to apply accurate ambient and directional lights, reflections and shadows in real time.
The lighting and shadows created by the MR360 toolkit contribute to the realism of this mixed reality scene.
Lighting and shadows are what sell realism and believability in the world of Mixed Reality (MR).
MR experiences create an environment that allows both physical and virtual objects to exist and interact in real-time.
For the artists and content creators of MR applications—who must successfully blend their digital creations in with physical surroundings—lighting is paramount. The more accurately they can replicate real-world lighting conditions in a 3D digital space, the more realistic and immersive the experience becomes for the end users of their products.
Content creators can spend hours adjusting the lights and reflection maps in their scenes to achieve the best realism but, until now, have not been able to achieve realistic lighting in real-time.
Victoria University of Wellington researchers have developed the MR360 toolkit to give content creators the power to build perfectly lit—and therefore incredibly realistic—mixed reality experiences using real-time360° video footage.
The tool automatically analyses live streamed 360° video to detect lighting conditions, allowing content developers to apply accurate, ambient and directional lights, reflections and shadows—and seamlessly blend the synthetic and real worlds together for a truly immersive, interactive experience.
The Wellington UniVentures team look forward to continuing to support Associate Professor TJ Rhee on his work to develop augmented virtual teleportation.
Features and benefits
Real-time light detection and estimation
Adaptable and automatic
Because the tool runs in real time, it adapts to the lighting conditions as the live stream video changes—no manual setup or precomputation required.
Cross-platform software
The MR360 toolkit has been developed to work with two of the most popular content creation engines on the market today—Unreal Engine 4 (UE4) and Unity.
Next steps
Wellington UniVentures is currently working with the project team to explore opportunities with mixed reality hardware and development tool businesses who may benefit from incorporating the technology into their software development kits.
The team is also seeking companies to test their light estimation method and discuss other MR innovations such as materials estimation.
For more information, please contact the Commercialisation Manager below.
