NASA Logo, National Aeronautics and Space Administration

+NASA Home

+Ames Home

Robust Software Engineering Project Wins AIAA Modeling and Simulation Best Paper Award
Intelligent Systems Division Banner

Robust Software Engineering Project Wins AIAA Modeling and Simulation Best Paper Award

The Adaptive Stress Testing of Aircraft Collision Avoidance Systems project paper titled "Differential Adaptive Stress Testing of Airborne Collision Avoidance Systems" (American Institute of Aeronautics and Astronautics - AIAA 2018-1923) has won the AIAA Modeling and Simulation Best Paper Award from the AIAA SciTech Forum and Exposition. The paper describes a novel approach for discovering scenarios where a failure occurs in the system under test, but not in a comparable baseline system. We analyze scenarios of Near Mid-Air Collisions (NMACs) where aircraft are equipped with either the next-generation Aircraft Collision Avoidance System v10 (ACAS X) prototype or the existing Traffic Alert and Collision Avoidance System (TCAS).

BACKGROUND: The Robust Software Engineering (RSE) group has been working with the Federal Aviation Administration (FAA) and the ACAS X development team to apply state-of-the-art NASA tools toward the validation and safety analysis of ACAS X. One of these tools is Adaptive Stress Testing (AdaStress), which is a software package that uses reinforcement learning to search for failure scenarios that are most likely to occur. We apply AdaStress to discover the most likely scenarios of NMAC in simulated aircraft encounters. Differential Adaptive Stress Testing (DAST) extends the AdaStress framework to find scenarios that occur in one system but not in another baseline system. In this paper, we apply DAST to find aircraft encounters that result in a NMAC when ACAS X is used, but do not result in a NMAC when the existing TCAS is used. The search is very challenging due to the large state space, multi time-step nature of the problem, and the exceptionally low probability of NMAC with ACAS X.

NASA PROGRAM FUNDING: System Wide Safety (SWS) project, Airspace Operations and Safety Program (AOSP), Aeronautics Research Mission Directorate (ARMD)

TEAM: Ritchie Lee (NASA/Carnegie Mellon University - CMU); CMU: Ole J. Mengshoel; JOHNS HOPKINS UNIVERSITY APPLIED PHYSICS LABORATORY: Jeffrey Brush, Ryan Gardner, Daniel Genin, and Anshu Saksena; STANFORD: Mykel J. Kochenderfer


First Gov logo
NASA Logo -