NTSB, FAA Spar Over Scope of License & Permit Evaluations

ntsb_logoBy Douglas Messsier
Managing Editor

The National Transportation Safety Board (NTSB) is concerned that the Federal Aviation Administration (FAA) might not be sufficiently addressing weaknesses in how it evaluates experimental permit and license applications submitted by commercial space companies.

The concerns involve the FAA’s response to one of eight recommendations the NTSB made in its final report on the crash of Virgin Galactic’s SpaceShipTwo in October 2014. NTSB investigators found shortcomings in the FAA’s evaluation and issuance of the experimental permit and a waiver under which flight tests of Sir Richard Branson’s suborbital space tourism vehicle were conducted.

To address the problems, the NTSB recommended that the FAA

direct Office of Commercial Space Transportation (AST) management to work with AST technical staff to (1) develop clearer policies, practices, and procedures that allow direct communications between staff and applicants, (2) provide clearer guidance on evaluating commercial space transportation permits, waivers, and licenses, and (3) better define the line between the information needed to ensure public safety and the information pertaining more broadly to ensuring mission success.

During its investigation, the NTSB found that FAA AST’s evaluators were not able to ask direct questions to the technical staff at Scaled Composites, which built SpaceShipTwo and tested it until the fatal crash.

Questions were filtered and “scrubbed” by agency management so Scaled’s technical staff would not have to deal with multiple calls from different evaluators. FAA officials have said this is a standard practice designed to reduce the regulatory burden on applicants.

Evaluators complained to NTSB investigators that the process made it difficult to get answers to key safety concerns. FAA AST management rejected some requests as outside the scope of the agency’s regulatory mandate. Evaluators said they sometimes received answers from Scaled that didn’t address the questions they had submitted.

In its response to the NTSB, the FAA denied there were any problems with communications, but said it had taken steps “to clarify and improve communications among the staff, management, and applicants during the permit and licensing evaluation process.

“AST technical staff members have always been, and will continue to be, engaged in direct communications with applicants on technical matter,” the FAA wrote.

“We found the opposite in our investigation of the SpaceShipTwo accident,” the NTSB responded. “We issued this recommendation because we found that, at the time of the evaluation of Scaled’s experimental permit applications, AST management underutilized AST evaluators’ expertise, even though the AST staff understood the risks associated with commercial space flight.

“Further, we found that the filtering of questions and the lack of direct communication between AST technical staff and Scaled technical staff impeded Scaled’s ability to take advantage of AST’s safety expertise,” the safety board said.

The NTSB also found that FAA AST lacked a clear dividing line between information required to protect the public and information needed to evaluate mission objectives.

“Certain aspects of a vehicle’s design and operation could impact both public safety and mission safety assurance,” the NTSB wrote. “Therefore, AST technical staff needs to fully understand the factors that might be critical to public safety, such as system failure modes and their effects, the potential for human errors that could contribute to a divergence from operating area containment boundaries, and hazard causes and controls.”

Under law, FAA AST has limited oversight of the emerging commercial space industry. There is a moratorium on promulgating regulations. The office’s main statutory authority is to protect the safety of the “uninvolved” public and third-party property.

In addition to its safety responsibilities, FAA AST also has the mandate to encourage the development of the nascent industry. Evaluators complained to NTSB investigators about pressure to finish permit reviews with the 120-day statutory limit and management’s desire to maintain good relations with applicants.

“AST management appeared to be more concerned with ensuring that the FAA’s authority was not being exceeded beyond defined limits and maintaining the time frame in which to approve experimental permit applications,” the NTSB noted.

In its response, the FAA said it has revised and consolidated its internal procedures for reviewing and issuing permit and license reviews. The new procedures include “comprehensive guidance for

  • the composition, management, and responsibilities of AST teams working in these areas, including such areas as coordination and communications within the team and with the applicant;
  • defining the statutory basis and conducting licensing and permit evaluations in accordance with the CST regulations and their associated FAA guidance and policy; and
  • resolution of technical issues within the team, and the elevation of these issues to AST management if they cannot be resolved at the team level.”

The FAA also noted that a management review board (MRB) makes all license, permit and waiver decisions.

“MRB meetings are open to all AST staff members and all AST staff and management are actively encouraged to raise any concerns associated with the license or permit review during MRB meetings,” the agency said.

FAA AST said it has been conducting a “comprehensive upgrade” of its safety management system (SMS) process “to more clearly articulate our safety policy, safety risk management, safety assurance, and safety promotion within the context of our legislative authority and regulatory framework for CST.

“The upgrade will also clarify the roles and responsibilities of AST’s management and staff in accordance with the recent improvements to AST operating practices and lessons learned. All AST personnel will receive training on the upgraded SMS,” the FAA added.

The agency indicated that its oversight would remain narrowly focused.

“It is critical that all AST staff members understand the FAA’s statutory authority is limited to the protection of public safety and property, national security and foreign policy interests of the United States,” the FAA wrote.

In response, the NTSB said it was encouraged by FAA AST’s decision to upgrade the SMS process.

“However, we are concerned by your statement that you are developing your SMS for AST because ‘it is critical that all AST staff members understand the FAA’s statutory authority is limited to the protection of public safety and property, national security and foreign policy interests of the United States,’” the NTSB said.

“We are concerned that the purpose of AST’s SMS will be to limit AST’s review to a narrowly defined interpretation of your statutory authority, and that such a narrow interpretation will not be consistent with our recommendation to better define the line between the information needed to ensure public safety and the information pertaining more broadly to ensuring mission success,” the safety board added.

The NTSB also questioned whether FAA AST’s new internal procedures are adequate to avoid past problems with poor communications and the pressure to finish evaluations within 120 days.

“Although these procedures address issues in this recommendation, we are concerned that the procedures may institutionalize the procedures and practices that we found in the SpaceShipTwo investigation,” the NTSB wrote.

The NTSB said the recommendation remains “open” pending the FAA’s response to its concerns.

The NTSB recommendation, FAA’s response and the NTSB’s subsequent response are reproduced below.

NTSB Recommendation No. A-15-25
Aug. 4, 2015

Direct Office of Commercial Space Transportation (AST) management to work with AST technical staff to (1) develop clearer policies, practices, and procedures that allow direct communications between staff and applicants, (2) provide clearer guidance on evaluating commercial space transportation permits, waivers, and licenses, and (3) better define the line between the information needed to ensure public safety and the information pertaining more broadly to ensuring mission success.

faa_logoFAA Response
Oct. 30, 2015

AST technical staff members have always been, and will continue to be, engaged in direct communications with applicants on technical matters. However, the FAA has recognized the need to improve the guidance and policy provided for our technical staff.

As a result, the FAA has taken steps to clarify and improve communications among applicants. AST also revised and consolidated its internal procedures for conducting reviews and issuing licenses and experimental permits with the release of P-002 in August 2015. Each of these procedures provides comprehensive guidance for:

  • The composition, management, and responsibilities of AST teams working in these areas, including such areas as coordination and communications within the team and with the applicant;
  • Defining the statutory basis and conducting licensing and permit evaluations in accordance with the CST regulations and their associated FAA guidance and policy; and
  • Resolution of technical issues within the team, and the elevation of these issues to AST management if they cannot be resolved at the team level.

Elevation to management is also required in cases where waivers, equivalent levels of safety, or the existing regulations, guidance or policy may not be sufficient due to novel aspects of proposed vehicle or its operations.

AST uses a Management Review Board (MRB) process when making license or permit determinations, including the need for any waivers or equivalent levels of safety findings. The MRB is made up of AST’s executive leadership, its division managers, and chief engineer. MRB meetings are open to all AST staff members and all AST staff and management are actively encouraged to raise any concerns associated with the license or permit review during MRB meetings.

It is critical that all AST staff members understand the FAA’s statutory authority is limited to the protection of public safety and property, national security and foreign policy interests of the United States.

To support this need, AST is also conducting a comprehensive upgrade of its Safety Management System (SMS) process to more clearly articulate our safety policy, safety risk management, safety assurance, and safety promotion within the context of our legislative authority and regulatory framework for CST.

The upgrade will also clarify the roles and responsibilities of AST’s management and staff in accordance with the recent improvements to AST operating practices and lessons learned. All AST personnel will receive training on the upgraded SMS. The revised SMS document and the training package are both targeted to be available by the beginning of Calendar Year 2016.

NTSB Response
Jan. 11, 2016

We are concerned by your statement that AST technical staff members have always been engaged in direct communications with applicants on technical matters. We found the opposite in our investigation of the SpaceShipTwo accident. We issued this recommendation because we found that, at the time of the evaluation of Scaled’s experimental permit applications,

AST management underutilized AST evaluators’ expertise, even though the AST staff understood the risks associated with commercial space flight. AST management appeared to be more concerned with ensuring that the FAA’s authority was not being exceeded beyond defined limits and maintaining the timeframe in which to approve experimental permit applications. Further, we found that the filtering of questions and the lack of direct communication between AST technical staff and Scaled technical staff impeded Scaled’s ability to take advantage of AST’s safety expertise.

We note that you recently developed and issued new internal operating procedures P 011 covering pre-application coordination with prospective applicants, and that you revised P 002 for conducting reviews and issuing licenses and experimental permits. You indicated that each of these procedures provides comprehensive guidance for the composition, management, and responsibilities of AST teams working in these areas, including such areas as coordination and communications within the team and with an applicant. The procedures also define the statutory basis and procedures used for conducting licensing and permit evaluations. The procedures also discuss the resolution of technical issues within the team, and the elevation of these issues to AST management if they cannot be resolved at the team level.

Although these procedures address issues in this recommendation, we are concerned that the procedures may institutionalize the procedures and practices that we found in the SpaceShipTwo investigation. We ask that you describe how P-011 and P-002 address the “filtering” and “scrubbing” of questions and the lack of direct communication between AST technical staff and the applicant’s technical staff, as well as the pressure on AST technical staff to complete their evaluation within 120 days even when there are unanswered technical questions that need to be answered before an appropriate evaluation can be completed.

In your letter, you indicated that AST is conducting a comprehensive upgrade of its safety management system (SMS). In our report about the SpaceShipTwo accident, we said that we were encouraged by AST’s progress in implementing SMS and we believed that, if SMS principles were followed, they would constitute an effective means for enhancing the regulatory oversight of the commercial space industry and would satisfy this recommendation.

However, we are concerned by your statement that you are developing your SMS for AST because “it is critical that all AST staff members understand the FAA’s statutory authority is limited to the protection of public safety and property, national security and foreign policy interests of the United States.”

In our report, we discussed that the dividing line between the questions that need to be asked to determine the risk to the public and those to assess mission objectives is not always apparent because certain aspects of a vehicle’s design and operation could impact both public safety and mission safety assurance. Therefore, AST technical staff needs to fully understand the factors that might be critical to public safety, such as system failure modes and their effects, the potential for human errors that could contribute to a divergence from operating area containment boundaries, and hazard causes and controls.

We are concerned that the purpose of AST’s SMS will be to limit AST’s review to a narrowly defined interpretation of your statutory authority, and that such a narrow interpretation will not be consistent with our recommendation to better define the line between the information needed to ensure public safety and the information pertaining more broadly to ensuring mission success.

As we discussed in our report, the development and use of an effective SMS will satisfy this recommendation. We ask that you respond to our concerns regarding P-011 and P-002, and the new SMS.

Pending your responses to those concerns, and completion of the SMS for AST, Safety Recommendation A-15-25 is classified OPEN—ACCEPTABLE RESPONSE.

 

 

  • Dave Huntsman

    In terms of requesting and getting needed data, versus making sure the companies are not continually “peppered with calls for info” is a dividing line, of course. And it’s not just with FAA/AST and the nascent commercial space transportation industry. Companies from traditional aerospace are wary of the extreme of allowing, say, any /all ol’ Federal employee to be able to call up any/all of their folks at any and all times and continually ask for “more data” et al – sometimes just out of curiosity. And has that ever happened? Sure; I’ve seen it myself in the past (on some of the more traditional programs; i.e., NASA cost-plus contracts, etc.).

    My feeling, tho, is that, at least leading up to the accident, AST may have leaned a little too far in the other direction (out of legitimate concern; not out of malice. I personally haven’t met a single AST employee, yet, who are real asses, tho they may exist). But, Normal, operational comm links can’t really have to go thru one (or more!) “management” filters – on both sides – without losing something in the process, and getting employees frustrated to boot. And contrary to AST’s response, frankly, the primary duty of management, a review board, etc. is not to ensure that employees understand AST’s “limitations” (statutory, or otherwise). Employees need to, IN ORDER:

    1. Protect the uninvolved public; and to take, recommend, communicate with anyone, push action, whatever, to see that occur;

    2. Encourage, facilitate, and promote the growth and competitiveness of the industry.

    3. Be very cognizant of the line of what, when it’s all said and done, they can seek to statutorily impose; while still keeping in mind that The Jobs are numbers 1 and 2 above.

    I can understand establishing management filters for even asking questions for data et al, IF there has been found to be a real problem with excess; i.e., multiple companies, say, complaining that too many folks from the FAA building have been calling/writing way too often. Then, one might set up a management review process – for a while – to establish some sort of discipline, to bring things under control. But, not forever. If you have to keep on having management filters forever, you either have the wrong people in place; or a company on the other end that doesn’t believe there should be a government regulator, at all.

    Prime, qualified people selected for working with each company/program should not have to go thru management bureaucratic hoops, in either direction, that ends up watering down what’s going on thru each hoop. Pick your leads with care, They have a primary job to do; and they should know, ultimately, where the real limits can be. Ensure that they not only do their immediate job, but establish relationships that are for the long-term – for their sake, for AST’s sake, and the company’s sake – and seek and be open to honest continuous feedback about how they are doing; but let them communicate. And when you need to change out a person, do so.

    But, strictly limiting any questions, via bureaucracy, that seems beyond an ultra-strict interpretation of what the ‘regulatory authority’, is its own slippery slope. After all, you only know that requirements, procedures, protocols, etc. are “safe enough”, when you’ve gone beyond that in your search for data, understanding, and checks.

    If AST hasn’t already in the past year, it would seem prudent to pick the prime inspectors, communicators, etc. with companies/programs with care – and then let them do their job, unhindered in most cases, until there is a problem of true (not theoretical) ‘over-doing it’. When the latter happens, either quickly perform an attitude adjustment on that individual, or replace him. But don’t frustrate an entire dedicated, even underpaid, professional staff, because one individual might ask one company one too many questions. That hurts everyone – including, by the way, the companies.

    We need smart, dedicated inspectors/regulators and, yes, people who understand they do have a duty to not overdue it; and who are so good, professional, and even useful, that the companies see them that way as well. That only happens if good professional people can have open communication back and forth, all the time.

  • Armchair Warlord

    In my line of work, if you don’t fulfill the regulator’s requirements, you don’t operate. And it is not the regulator’s business to help you out.