NASA Logo, National Aeronautics and Space Administration


In this project, we develop a framework for systematic benchmarking of diagnostic technologies. The diagnosed system is the Electrical Power System (EPS) testbed in the ADAPT lab at NASA Ames Research Center. Our benchmarking approach shows how to empirically generate realistic data sets for diagnostic benchmarking, in the context of ADAPT EPS, and emphasizes the use of standardized vocabularies and protocols which together enable “apples to apples” assessments of the effectiveness of different diagnostic technologies.

The framework introduces a number of specifications including a standardized fault catalog, a common set of metrics, a library of modular and standardized test scenarios, a test protocol, and an evaluation algorithm for processing diagnostic data and calculating the performance metrics. The testing procedure is usually scenario-based, where each scenario may have faults injected into the system. To detect faults, each diagnostic algorithm has access to real-time data from the ADAPT EPS. Moreover, a standardized output scheme is enforced on the diagnostic algorithms to ensure the generation of common data sets for the calculation of metrics. The data from the testbed and this output of the diagnostic system are saved to a database, and the diagnostic algorithm performance is evaluated according to a predefined set of metrics.

adapt lab

The EPS testbed in the ADAPT lab at Ames Research Center.


Project Lead:
Scott Poll

David Nishikawa
Adam Sweet

Past Members
Eric Barszcz
Joe Camisa
David Garcia
David Hall
David Iverson
Tolga Kurtoglu
Charles Lee
Dougal Maclise
Ole Mengshoel
Christian Neukom
John Ossenfort
Gerald Temple
Serge Yentus

Past Interns
Matt Daigle
Nigusu Ergete
Zachary Green
Craig Harrison
David Jensen
Bradley Knox
Cyril Lan
Robert Niffenegger
Godohaldo Perez
Brian Ricks
Indranil Roychoudhury
Trevor Tetzlaff
Michael Wilson
Stephanie Wright
Justin Yu

First Gov logo
NASA Logo -