SpaceX Accident Investigation Update
January 2, 2017, 9:00am EST
Over the past four months, officials at the Federal Aviation Administration (FAA), the U.S. Air Force (USAF), the National Aeronautics and Space Administration (NASA), the National Transportation Safety Board (NTSB), along with several industry experts, have collaborated with SpaceX on a rigorous investigation to determine the cause of the anomaly that occurred September 1 at Space Launch Complex 40 (SLC-40) at Cape Canaveral Air Force Station in Florida. This investigation team was established according to SpaceX’s accident investigation plan as approved by the FAA. As the primary federal licensing body, the FAA provided oversight and coordination for the investigation.
Investigators scoured more than 3,000 channels of video and telemetry data covering a very brief timeline of events – there were just 93 milliseconds from the first sign of anomalous data to the loss of the second stage, followed by loss of the vehicle. Because the failure occurred on the ground, investigators were also able to review umbilical data, ground-based video, and physical debris. To validate investigation analysis and findings, SpaceX conducted a wide range of tests at its facilities in Hawthorne, California and McGregor, Texas.
The accident investigation team worked systematically through an extensive fault tree analysis and concluded that one of the three composite overwrapped pressure vessels (COPVs) inside the second stage liquid oxygen (LOX) tank failed. Specifically, the investigation team concluded the failure was likely due to the accumulation of oxygen between the COPV liner and overwrap in a void or a buckle in the liner, leading to ignition and the subsequent failure of the COPV.
Each stage of Falcon 9 uses COPVs to store cold helium which is used to maintain tank pressure, and each COPV consists of an aluminum inner liner with a carbon overwrap. The recovered COPVs showed buckles in their liners. Although buckles were not shown to burst a COPV on their own, investigators concluded that super chilled LOX can pool in these buckles under the overwrap. When pressurized, oxygen pooled in this buckle can become trapped; in turn, breaking fibers or friction can ignite the oxygen in the overwrap, causing the COPV to fail. In addition, investigators determined that the loading temperature of the helium was cold enough to create solid oxygen (SOX), which exacerbates the possibility of oxygen becoming trapped as well as the likelihood of friction ignition.
The investigation team identified several credible causes for the COPV failure, all of which involve accumulation of super chilled LOX or SOX in buckles under the overwrap. The corrective actions address all credible causes and focus on changes which avoid the conditions that led to these credible causes. In the short term, this entails changing the COPV configuration to allow warmer temperature helium to be loaded, as well as returning helium loading operations to a prior flight proven configuration based on operations used in over 700 successful COPV loads. In the long term, SpaceX will implement design changes to the COPVs to prevent buckles altogether, which will allow for faster loading operations.
SpaceX is targeting return to flight from Vandenberg’s Space Launch Complex 4E (SLC-4E) with the Iridium NEXT launch on January 8. SpaceX greatly appreciates the support of our customers and partners throughout this process, and we look forward to fulfilling our manifest in 2017 and beyond.
Editor’s Note: So, this is a two-step solution. In the interim, they’re still flying with helium tanks that could have these buckles (voids) in them. But, they believe that warmer helium and changes in the helium loading procedures will prevent another explosion. The long term solution is to redesign the COPV’s to prevent the buckles.
In essence, SpaceX doesn’t want the Falcon 9 grounded while they redesign and test the COPV’s. The question is how long is that going to take? How many flights are there going to be with COPV’s inside the LOX tank that may have the same buckles in them?
The other question is this: SpaceX seems to be saying they were using a different helium loading procedure when the rocket was destroyed than they had used previously. Was it just for this one occasion? Had they used it on previous flights? It would astonish me if they were testing some new procedure for the first time with a $195 million satellite atop of the booster.
I’d also note that I don’t see any reference to probable cause. Instead they’re talking about “credible” causes. I also don’t see any reference to the FAA signing off on the launch next week.