NASA Logo, National Aeronautics and Space Administration

+NASA Home

+Ames Home

T2 Augmented Reality Demonstration Presented at NASA/Microsoft Solutions Workshop
Intelligent Systems Division Banner

T2 Augmented Reality Demonstration Presented at NASA/Microsoft Solutions Workshop

The Autonomous Systems and Operations (ASO) project’s Dr. Jeremy Frank presented the T2 Augmented Reality (T2AR) demonstration during the latest in a series of NASA/Microsoft Solutions Workshops. T2AR is the first in-space operational use of the Microsoft HoloLens (a self-contained holographic computer) in combination with custom-built Augmented Reality (AR) software, which enables astronauts to perform unassisted maintenance and inspection procedures on major pieces of crew-support hardware. In this groundbreaking demonstration, T2AR:

  • Used 3D directional cues to direct the astronaut to the proper work sites
  • Displayed the procedure (and could even read and follow verbal instructions)
  • Laid AR cues over the hardware as appropriate for the procedural step being performed
  • Provided supplemental info, like instructional videos and system schematics, to assist in performing the procedure

Operational Firsts

  • First autonomous crew use of AR technology (‘untethered’ from Mission Control)
  • First use of AR to perform maintenance and inspection of International Space Station (ISS) systems in space

Technical Firsts

  • First use of commodity Microsoft HoloLens on orbit
  • First use of NASA-built HoloLens application for maintenance and inspection of ISS systems

For more information on T2AR, see the NASA Public Affairs Office story on nasa.gov.

BACKGROUND: Future astronauts at greater distances from Earth will need smarter tools employing Artificial Intelligence (AI) techniques to make decisions without inefficient communication back and forth with ground-based mission control. The Autonomous Systems and Operations (ASO) project develops advanced technology to enable Exploration missions.

NASA and Microsoft have now run three Solutions Workshops with a focus around the Microsoft Azure Cloud in Action. The NASA Point of Contact for these workshops is Jeanne O’brien, jeanne.obryan@nasa.gov. The most recent agenda included the following topics:

  • HoloLens and use cases
  • Extreme edge computing and analytics – Azure in space
  • Azure cloud successes
  • Planetary computer – using NASA data for a better world
  • NASA Enterprise Managed Cloud Computing (EMCC) model

NASA PROGRAM FUNDING: Advanced Exploration Systems (AES) Program, Human Exploration and Operations Mission Directorate (HEOMD)

TEAM: AMES RESEARCH CENTER (ARC): Jeremy Frank and Chris Knight; JOHNSON SPACE CENTER (JSC): Jeffery Mauldin, Truyen Van Le, and Lui Wang

POINT OF CONTACT: Jeremy Frank, jeremy.d.frank@nasa.gov

First Gov logo
NASA Logo - nasa.gov