NASA Logo, National Aeronautics and Space Administration
Intelligent Systems Division Banner


Sponsored by
the Safe Autonomous System Operations (SASO) project
in the ARMD Airspace Operations and Safety program

Organized by SASO researchers

Date: January 6th and 7th, 2016
Location: Collated with AIAA Scitech 2016, San Diego, CA

In the near future, autonomy will play an important role in civil aviation, and its applications will range from vehicles and platforms (UAVs, transport-class, including supersonic to hypersonic, aircraft) to airspace operations, or health management systems. This infusion of autonomy is driven by a need of optimizing airspace operations to accommodate increasing traffic density (e.g., adaptive trajectory-based operations, autonomous tugs, close parallel runways, and dynamic separation assurance), reducing operation costs to ensure that US operators can compete with emergent countries, and enabling new business models (e.g., fire fighting, UAS-based package delivery and precise aerial photography). In essence virtually every component of the National Airspace System will become increasingly autonomous. Yet we need to do so in a safe manner and have techniques and processes in place to ensure the safety of the public.

Autonomous systems are characterized by their ability to make and execute decisions without human intervention and by attributes such as self-configuration, self-optimization, self-protection and self-healing. These systems present new assurance challenges, including the following questions:

  • Can current assurance methods address autonomy? Where are the limits and can we go beyond them? Should we limit the degree of autonomy when considering safety?
  • Do we really need to sacrifice performance for assurance? Can we drive towards a notion of peformance-based assurance?
  • How can we reason about human interaction with autonomous systems, especially when the autonomous systems hands over control to the human?
  • How do we provide assurances for existing systems, especially when they were not originally developed to the required design assurance levels?
  • Can we address assurance challenges in less time than it takes today? Can it be done without requiring a high-level of analytical skill on the part of the practitioner?

The purpose of this workshop is to identify where we are about increasingly autonomous system assurance, where the research gaps are, and how NASA can work with all stakeholders to provide assurance techniques that preserve the performance benefits of autonomy. This workshop is for participants who represent industry (manufacturers, retailers, services delivery operators, air transportation expert, system designers and integrators), university researchers, government organizations, and subject matter experts. Collaborations are anticipated to develop, and demonstrate assurance techniques for increasingly autonomous systems.


Block 1 - Wednesday 9am to 12:30pm - Describing Autonomy for System Assurance

In this session, we survey the current state of assurance in autonomous systems, as well as provide a commentary on the state of the art in terms of practices used to achieve assured autonomy. Topics that will be addressed include a discussion of this year’s Workshop on Certification of Non­Deterministic Systems, the recent NRC report on Autonomy Research for Civil Aviation, along with discourse on current practices in the UAS industry as well as at NASA’s Autonomy Incubator.

We begin with a panel discussion to explore these areas, via an interactive Q&A session with the audience. The audience questions will be used to drive and derive directions for investigation that will be captured by the moderators in the report­out. This will be followed by several example lightning­-fast (5­-minute) talks on new ideas for describing and generating requirements for autonomous systems.

Panelists: B. Danette Allen (NASA Langley), Andy Thurling (AeroVironment), Andy Lacher (MITRE), Ella Atkins (University of Michigan), Lael Rudd (Lockheed Martin Advanced Technology Labs)

Lightning­-fast Talks: Corey Ippolito (NASA Ames), Devesh Bhatt (Honeywell), Mike Lowry (NASA Ames)

Block 2 - Wednesday 2pm to 5:30pm - Methods for Enabling Autonomy

In this session, we discuss current and upcoming techniques that are driving autonomous system development in aviation and the need for new assurance techniques in order to enable greater assured functionality for autonomous systems. Topics that will be addressed include the design, manufacture, fielding, maintenance and retirement of autonomous systems, including relevant elements such as COA/Certification and regulatory approval. Architectures for autonomy will also be discussed.

We begin the session with subject matter expert talks, addressing the issues inherent in designing and fielding multiple types of autonomous platforms with differing mission capabilities and assurance levels. We will then form moderated breakout groups organized by tools, techniques, capabilities and barriers.

Speakers (2­ - 3 pm): John Valasek (Texas A&M, Director of the Center for Autonomous Vehicles and Sensor Systems), Eric Johnson (Georgia Tech, Lockheed Martin Associate Professor of Avionics Integration)

Breakout Session (3:15 ­- 5:30pm). We request that the audience help us in exhaustively exploring the capabilities uniquely enabled by autonomy in aviation systems, the current new hardware and mathematical and computational techniques that are enabling this autonomy, and the barriers to assurance. The results of the breakout will be used to generate a report regarding critical needs as well as competencies required to enable assured autonomy. We will also schedule Thursday lightning­-fast talks for audience members who are inspired by the day’s proceedings.

Block 3 - Thursday 9am to 12:00pm - Managing Key Issues for Assured Autonomy

In this session, we examine several key challenges that directly impact our ability to generate assurance arguments for increasingly autonomous systems. Topics that will be addressed include human­ machine interaction under increasingly autonomous systems, the management of uncertainty, and the management and mitigation of communications criticality.

We will begin with subject matter expert talks highlighting the effects of human­ machine teaming on autonomous systems. We then proceed to a panel exploring the issues of uncertainty management and mitigating the effects of communications criticality. We then culminate with as set of lightning-­fast (5­-minute) talks for all assurance of autonomy themes. We strongly encourage these lightning-­fast talks from our participants — please sign up in the Wednesday sessions!

Speakers: Steven Young (NASA Langley Research Center)

Panelists: Kerianne Gross (AFRL), Johann Schumann (NASA Ames/SGT), Jim Murphy (NASA Ames), Natalia Alexandrov (NASA Langley), Mats Heimdahl (University of Minnesota)

Lightning­-fast talks: Kalmanje Krishnakumar (NASA Ames), Mark Skoog (AFRC), other speakers from the attendees

Block 4 - Thursday 2pm to 5:30pm - Assurance Tools and Techniques for Trusted Autonomy

In this session, we explore new concepts and methods to facilitate the Verification and Validation (V&V) and Certification of increasingly autonomous systems. Topics that will be addressed include tools and techniques that can be used to assess and assure Safety and Security, as well as engender Trust in increasingly autonomous systems, on the behalf of designers, evaluators, users and the general public will be investigated.

We begin with a panel discussion to address these issues, via an interactive Q&A session with the audience. The audience questions will be used to drive and derive directions for investigation that will be captured by the moderators in the report­-out. We will then form moderated breakout groups based on the previously identified barriers to assurance, and then identify promising research and development areas.

Panelists (2 - 3 pm): Lee Pike (Galois), Natarajan Shankar (SRI), Cristoph Torens (DLR German Aerospace Center), Darren Cofer (Rockwell Collins), Irene Gregory (NASA Langley)

Breakout Session (3:15­ - 5:30pm): We ask that the audience revisit the barriers generated during the previous day and identify promising technologies for autonomy assurance.

First Gov logo
NASA Logo -