Since its release in 2016, Microsoft HoloLens and Mixed Reality have captured significant interest from NASA. The HoloLens, paired with Mixed Reality, is shaping up to be the ideal solution for addressing various needs, ranging from remote operations and self-inspections to maintenance and repair tasks. It’s becoming increasingly evident that the HoloLens and Mixed Reality are the right tools for the job.

Communication is absolutely crucial for NASA, especially in the context of space operations. When astronauts are in space, they can’t always rely on immediate assistance from subject matter experts. The time delay in communication can mean the difference between successful repairs and the accidental damage of expensive equipment, costing millions of dollars. To combat this challenge, NASA is actively developing tools to empower astronauts to operate spacecraft and systems autonomously, without requiring constant external guidance.

One of the notable projects in this regard is the T2 Augmented Reality (T2AR) initiative. This project marks the “first in-space operational use of the HoloLens in combination with custom-built AR software.” The software itself is pioneering, enabling astronauts to perform self-inspections, maintenance, and repairs on equipment that would traditionally demand assistance from experts on the ground. This signifies a significant shift in operations, where astronauts can diagnose and address equipment issues independently, even without prior knowledge.

T2AR effectively combines the functionalities of HoloLens applications like Dynamics 365 Guides and Remote Assist. Astronauts wear the HoloLens, which provides a hands-free, heads-up display for guiding them through tasks. Step-by-step instructions and visual cues help astronauts conduct inspections, diagnose problems, and carry out repairs. In situations where communication is compromised, astronauts possess the tools to solve real-time issues on their own. When communication is possible, they can still consult subject matter experts back on Earth for support. These experts can identify specific components and provide guidance on fixing issues as if they were physically present.

Early outcomes from the T2AR project are promising. NASA and astronauts aboard the International Space Station have successfully utilized the HoloLens and T2AR to perform maintenance on the T2 Treadmill, an exercise equipment. The HoloLens displays holographic PDF documents within the astronaut’s field of view, allowing for a hands-free experience. With the assistance of the T2AR software, astronauts receive step-by-step instructions and cues, enhancing their work process.

The success of this initiative has led NASA to explore the application of this solution in other operational areas. For instance, scenarios on the moon or Mars, where communication delays or disruptions are anticipated, could greatly benefit from this technology. Such implementation would enable astronauts to be self-reliant and fully operational, even in challenging environments where external guidance is limited.