by Douglas Messier
NASA’s X-59 Low Boom Flight Demonstrator (LBFD) Project is progressing well toward its first flight test at the end of 2021 or early 2022 even though its cost has increased and schedule has slipped, according to a new audit by the space agency’s Office of Inspector General (IG). (Full Report)
The ambitious project will test designs and techniques for reducing the sonic booms caused supersonic aircraft. If successful, the research would allow a new generation of supersonic transports to fly over land rather than being confined to over-ocean travel as the now-retired Concorde airplane was it carried passengers from 1976 to 2003.
Despite challenges, the project completed a successful Critical Design Review (CDR) last September that determined that prime contractor Lockheed Martin could proceed to full-scale fabrication, assembly, integration and testing.
“The Project Review Team (Team) assessed the Project’s progress in accordance with relevant CDR-related criteria,” the audit found. ” In December 2019, the Team reported favorably on the Project’s progress, noting that it exceeded expectations for this stage in the life cycle and that the completed analysis work was high quality and provided a sound basis for ensuring the design satisfied airworthiness and mission requirements. They also found that the technical design of the aircraft was sound and, in most cases, matured to or beyond what was expected at CDR.”
NASA awarded a contract to Lockheed Martin in March 2018 to build the experimental aircraft. The project’s $583 million budget “included 13 months of schedule reserve and $134 million in cost reserve to build and complete low-boom acoustic validation flights by October 2023,” the audit stated.
“However, the cost and schedule baselines have been negatively impacted by a 5-week government shutdown from December 2018 to January 2019 and reassignment of a test location,” the report added. “Overall, the contract is estimated to cost $37 million more at completion than originally expected and Lockheed Martin is 2 months behind its planned schedule due to challenges in hiring qualified technical personnel.”
Lockheed Martin is taking steps to get the program back on schedule. However, the on-going work slowdown caused by COVID-19 pandemic is likely to hinder these efforts.
The audit found that Lockheed Martin and LBFD project managers experienced difficulty in implementing an Earned Value Management System (EVMS), a management tool for measuring and assessing the project’s performance and process.
The problem was that Lockheed Martin’s facility in Palmdale, Calif., lacked an EVMS certified by the Defense Contract Management Agency (DCMA) as called for in the contract. NASA did not realize this due to “clerical and database errors,” the audit stated.
“As a result LBFD managers expended a substantial amount of time and effort verifying the reliability and accuracy of EVM data provided by Lockheed Martin,” the report added. “Consequently, the Agency will spend approximately $130,000 for DCMA to perform surveillance and certification testing at the Palmdale location.”
The audit also praised NASA for how it handled the acquisition process.
“LBFD is the Aeronautics Research Mission Directorate’s (ARMD) first attempt at a large-scale (over $250 million) development project,” the report said. “We found that LBFD management instituted a sound acquisition strategy when Lockheed Martin was issued a task order under an existing contract for the preliminary design of the aircraft and was then selected as the contractor for LBFD’s subsequent phases after NASA conducted a full and open competition.”
The audit’s Results in Brief section follows.
Management of the Low-Boom
Flight Demonstrator Project
NASA Office of Inspector General
May 6, 2020
Results in Brief
Why We Performed This Audit
Within the next 20 years, the projected growth of passenger air travel worldwide will require revolutionary improvements in aircraft performance and technology to make flying more safe, efficient, scalable, and environmentally friendly. NASA’s aeronautics programs are working on near-term solutions to address several of these challenges by developing innovations in commercial supersonic aircraft.
Since 1973, the Federal Aviation Administration (FAA) has banned commercial aircraft from flying overland within the United States at supersonic speeds—faster than the speed of sound—due to the loud sonic boom the aircraft produces.
In March 2018, NASA contracted with the Lockheed Martin Corporation (Lockheed Martin) to develop a single experimental aircraft or X-plane known as the Low-Boom Flight Demonstrator (LBFD) that produces a quieter sonic boom while flying at supersonic speeds.
The Agency is building the aircraft to develop a database of community responses to overland supersonic flights for use by the FAA and International Civil Aviation Organization (ICAO) at a planned 2025 meeting to support development of a new noise-based standard for supersonic overland flight.
In this audit, we examined whether NASA was effectively managing the LBFD Project to accomplish its technical objectives while meeting established milestones and controlling costs. Specifically, we reviewed the Project’s
(1) acquisition and contract management,
(2) project management and risk,
(3) Joint Cost and Schedule Confidence Level (JCL) estimate,
(4) Earned Value Management (EVM), and
(5) lessons learned from NASA and other federal agencies’ acquisitions.
To complete this work, we reviewed the acquisition, management, and oversight of Lockheed Martin project staff along with the effectiveness of the EVM program and implementation of JCL; analyzed LBFD-specific documentation from NASA, the LBFD Project, and Lockheed Martin; examined the LBFD contract with Lockheed Martin; and interviewed LBFD Project personnel, other NASA officials, and key staff at Lockheed Martin.
What We Found
LBFD is the Aeronautics Research Mission Directorate’s (ARMD) first attempt at a large-scale (over $250 million) development project. We found that LBFD management instituted a sound acquisition strategy when Lockheed Martin was issued a task order under an existing contract for the preliminary design of the aircraft and was then selected as the contractor for LBFD’s subsequent phases after NASA conducted a full and open competition.
The LBFD Project also implemented an innovative project management structure that leveraged geographically dispersed aeronautics expertise across multiple NASA Centers rather than designating a single Center as the lead for LBFD development. In addition, the Project provided the contractor more-than-expected amounts of government furnished equipment (GFE) that saved procurement costs.
LBFD is also the first ARMD project required to develop a JCL analysis (a statistical estimating tool that produces a cost and schedule baseline to predict the probability of a project being completed at a certain cost and on a certain schedule).
The Project’s JCL resulted in an Agency Baseline Commitment of $583 million that included 13 months of schedule reserve and $134 million in cost reserve to build and complete low-boom acoustic validation flights by October 2023.
However, the cost and schedule baselines have been negatively impacted by a 5-week government shutdown from December 2018 to January 2019 and reassignment of a test location. Overall, the contract is estimated to cost $37 million more at completion than originally expected and Lockheed Martin is 2 months behind its planned schedule due to challenges in hiring qualified technical personnel.
Thus, NASA needs to ensure contingency plans are in place to account for flight or data collection delays to guarantee delivery of the database of community responses to the ICAO in 2025.
We also found Lockheed Martin and LBFD Project managers experienced difficulties implementing EVM, a project management tool for measuring and assessing a project’s performance and progress.
The NASA FAR Supplement requires the contractor to use an Earned Value Management System (EVMS) and provide documentation showing compliance with EVM guidelines and also specifies that the Agency shall request the Defense Contract Management Agency (DCMA) determine the adequacy of proposed EVMS plans and system compliance.
NASA used an online database rather than directly request DCMA to verify Lockheed Martin’s EVMS certification. Due to clerical and database errors, NASA discovered Lockheed Martin’s Palmdale, California, location did not have a DCMA-certified EVMS as believed, and as a result LBFD managers expended a substantial amount of time and effort verifying the reliability and accuracy of EVM data provided by Lockheed Martin. Consequently, the Agency will spend approximately $130,000 for DCMA to perform surveillance and certification testing at the Palmdale location.
What We Recommended
To ensure Low-Boom Flight Demonstration Mission success, increase accountability for future X-plane developments, improve EVM-related processes and reporting, and improve DCMA involvement with NASA contracts , we made the following recommendations to NASA management:
(1) finalize the schedule-driven contingency plan for the community response testing and account for the personnel and costs it will require to implement;
(2) perform a cost-benefit analysis of implementing internal EVM during Phase D of LBFD development;
(3 and 4) document and provide both the project management approach and JCL analysis approach used by LBFD management to the NASA Chief Knowledge Officer;
(5) establish a process to be used during source evaluation boards and source selections that includes direct contact with the Center EVM Working Group Representative and cognizant DCMA office to verify all contractor proposed information related to EVM;
(6) establish NASA requirements for programs and projects to perform internal EVM reporting that follow the same timeline as contractor reporting;
(7) evaluate whether the monetary threshold for performing internal EVM is sufficient or additional criteria would be beneficial regarding the dollar-value of tasks related to providing GFE and performing in-house development work compared to NASA personnel performing integration, review, and management functions; and
(8) provide information and training on the availability, use, and responsibilities of DCMA during source evaluation boards and source selections.
We provided a draft of this report to NASA management who concurred with six of our eight recommendations and described corrective actions it will take. We consider management’s comments to those recommendations responsive and therefore, the recommendations are resolved and will be closed upon completion and verification of the proposed corrective actions.
Management did not concur with Recommendations 6 and 7 related to adjusting requirements and implementing additional criteria for in-house EVM reporting. These recommendations will remain unresolved pending further discussion with the Agency.