Current Mixed Reality Head-Mounted Displays Are Unsuitable for Pilot Training, Stanford University Study Reaffirms

Stanford University recently released a study that examined the psychological consequences of current mixed reality (MR) headsets—a type of extended reality (XR) technology that is increasingly appearing in consumer products, including Apple Vision Pro and Meta Quest 3, as well as in pilot training. According to the study, current MR headsets with video passthrough, which allow users to interact with their environments while engaging with digital content displayed on a headset, can cause significant visual and spatial distortions and motion sickness. As such, the study’s authors urged caution against prolonged or professional use. These findings reflect important implications for pilot training: current MR is not a suitable option, and virtual reality (VR) remains the best and only XR solution for immersive, hands-on training and for fulfilling civil aviation regulatory requirements.

There are three types of XR technologies, all increasingly present in pilot training. Therefore, it is imperative that aviation leaders can discern the capabilities and limitations of each—an educational effort led by the International Civil Aviation Organization (ICAO).

As presented by the ICAO, the three types of XR technologies in head-mounted displays (HMDs) used for pilot training include augmented reality (AR), which overlays computer-generated visuals onto the real world via transparent lenses; virtual reality (VR) HMDs, which exclusively display computer-generated visuals; and mixed reality (MR) HMDs, which blend AR and VR to allow select real-world visuals to pass through front-facing cameras onto headset screens, enabling users to interact with virtual and real environments simultaneously.

Loft Dynamics is a pioneer of VR simulators, creating fully immersive and highly realistic digital environments to train pilots effectively and safely – without any of the negative impacts of current MR outlined in Stanford’s study. It remains the first and only XR training device to receive regulatory qualification. Since, the company’s technology has gained industry-wide adoption and recognition around the world. Examples include the European Union Aviation Safety Agency, which mandates the use of its simulators for pilot training instead of an aircraft whenever possible; the Federal Aviation Administration, which has installed its simulators to advance aviation goals; and organizations such as the Los Angeles Police Department’s air support division and the prestigious test pilot school EPNER—both of which demand a diverse range of training capabilities.

"While VR is more challenging to build, the resulting quality of training achieved is unparalleled. The decision to use VR, rather than MR, was obvious from the start and is corroborated by the fact that VR remains the only XR technology authorized by aviation regulators," said Fabi Riesen, founder, and CEO of Loft Dynamics. "Pilot training has evolved significantly from using visual dome projections to XR head-mounted displays. However, as this study's findings reaffirm, it's critical to reassess the use of any MR in applications where faithful representation of the real world is imperative—regardless of how frequently or infrequently MR is used."

• Pass-through Latency: Current MR pass-through latency is too high for two people to high-five each other, making it unsuitable for the quick motion required on a dynamic simulator.
• Visual Distortion: Head rotations and body movements caused MR users to experience video distortions, wherein stationary objects appeared to move, stretch, or warp, affecting tasks such as drawing and concentration.
• Distance Estimation: MR users struggled to accurately judge distances in various scenarios, including spatial tasks, estimating space dynamics, and properly executing certain motor functions.
• Aftereffects and Visual Adaptation: MR users experienced errors in reaching or perceiving spatial relationships accurately, impacting activities such as walking, talking, gesturing, and driving.
• Simulator Sickness: MR users experienced symptoms of simulator sickness, including eye strain, nausea, dizziness, and headaches, due to the mismatch between the virtual content and the real-world video pass-through.

"Before anyone continues to use current MR, or even thinks about qualifying current MR for use in pilot training, they must refute the study's findings. The reported impacts could lead to negative training—a decline in hand-eye coordination when operating controls and instruments, increased errors in perception and action, prolonged adaptation periods that could jeopardize flight safety during critical maneuvers, misjudgment of distances, and difficulty adjusting to real-world flight scenarios after using current MR technology," Riesen adds. "These findings should only strengthen authorities' continued conviction that VR stands as the sole qualifiable XR technology for pilot training. I urge developers of pilot training solutions to prioritize VR over current MR for safer and higher-quality training. This transcends our company; it's about collectively and safely cultivating and expanding the pipeline of proficient pilots, especially as the world races to solve the pilot shortage."

Loft Dynamics specializes in designing, manufacturing, and operating VR flight simulation training devices (FSTDs) for various aircraft systems. Founded in 2016, our mission is to revolutionize global aviation by providing best-in-class pilot training that is safe, accessible, and affordable. Our immersive simulators, equipped with a 3D high-resolution panoramic view, dynamic six-degrees-of-freedom motion platform, and a full-scale replica cockpit with a unique pose tracking system, are 20 times less expensive and 10 times smaller than traditional flight simulators. We provide continuous and comprehensive customer support, from installation and maintenance to training and regulatory approvals, allowing users to focus on core business operations. Leading manufacturers, airlines, operators, schools, and organizations worldwide choose Loft Dynamics simulators—the world's first and only VR FSTDs with EASA FTD Level 3 regulatory approval. Based in Zurich, Switzerland, our team comprises passionate engineers, developers, and aviation experts.