NASA Logo, National Aeronautics and Space Administration

AdaStress - Adaptive stress testing

ACAS-X example for Adastress AdaStress is a software package written in Julia for the intelligent stress testing and explanation of safety-critical systems. The tool provides two major functionalities that can be used in tandem or separately. Adaptive stress testing (AST) is an accelerated simulation-based stress testing method for finding the most likely path to a failure event; and grammar-based decision tree (GBDT) can analyze a collection of these failure paths to discover data patterns that explain the failure events. AdaStress has been successfully applied to stress test a prototype of the next-generation Airborne Collision Avoidance System (ACAS X), where it found and analyzed aircraft encounters that result in near mid-air collision.

Adastress Architecture Adaptive stress testing (AST) is a new and improved approach to simulation-based stress testing that finds the most likely path to a failure event. Rather than an undirected search, as is done in the traditional Monte Carlo sampling approach, AST formulates stress testing as a sequential decision process then optimizes it using reinforcement learning algorithms. The key idea is to define a reward function to give evaluative feedback to the reinforcement learner, so that it can automatically discover important parts of the state space on its own. The reward function is designed to find failure events as the primary objective and to maximize the path probability as a secondary objective.

Key Benefits

  • No need for access to the full system state - Instead, the algorithm can use control of the pseudorandom seed as a substitute.
  • Interpretable classification model – in the form of a decision tree that can be used to predict whether a failure event occurs.
  • Explainability – categorize data and groups similar failure events together and then explains the unique properties of each group in the form of logical expressions derived from the user-supplied grammar.

Applications

Studies with ACAS-X and autonomous cars.

References

Updated December 2017


Active Members

Ritchie Lee

University Collaborators

Ole Mengshoel (CMU)
Mykel Kochenderfer (Stanford University)

First Gov logo
NASA Logo - nasa.gov