Webb Space Telescope Problems: Excessive Voltage, Wrong Solvent & Loose Nuts

Artist’s impression of James Webb Space Telescope. (Credit: NASA)

NASA established the Independent Review Board (IRB) in April to evaluate the space agency’s $8 billion James Webb Space Telescope (JWST). The report, released last month, revealed a number of eye opening details about problems that NASA and the prime contractor, Northrop Grumman Aerospace Systems (NGAS), have been experiencing in building the telescope and managing the program. Below are some key excerpts.

Independent Review Board Report
NASA James Webb Space Telescope

Full Report

Selected Excerpts

There have been several JWST Project problems due to human-induced errors that had substantial cost and schedule impact. In one case, an improper solvent was used to clean propulsion system valves that had been stored. The error was a failure to check with the valve vendor to ensure the solvent to be used was recommended and would not damage the valves. The valves had to be removed from the spacecraft, repaired or replaced, and reinstalled.

Another human-induced error was improper test wiring that caused excess voltage to be applied to transducers. The error resulted from an improper interpretation of a process step. The error should have been detected by the inspector, who did not inspect, but relied on the technician’s word that he had done the wiring correctly.

To address a risk that fasteners for sunshield membrane covers might snag the membrane, the fastening lock nuts were tightened only to be flush with their bolts. Unfortunately, this compromised the locking mechanism, and after the test, loose hardware was found in the lower area of the spacecraft.

The Webb IRB recommends that NGAS review processes/procedures to be sure they are current, accurate and implementable. The Webb IRB has noted two examples of human induced errors resulting from incomplete procedures (e.g., requiring that the solvent selected for cleaning valves be recommended by the vendor) or processes that could be misinterpreted (e.g., the improper wiring that caused excess voltage to be applied to transducers). Both examples resulted in substantial schedule and cost increases.

The Webb IRB recommends that individuals at the working level be trained to understand that, on JWST, seemingly small errors produce large consequences.

The Webb IRB recommends that individuals scheduled to perform tasks on the observatory be properly trained, capable, and certified to perform those tasks and properly supervised. Those not properly certified must not be allowed to perform tasks that could result in damage to the system. This is an especially critical discipline to maintain, particularly on second or third shift activities and when schedule pressure exists (which will likely always be the case on JWST Integration & Test (I&T) and when supervision may be thin.

The Webb IRB has noted lapses in individual accountability that have substantially increased schedule and cost and, potentially, increased risk. The Webb IRB recommends that discipline be continually instilled in the workforce such that people feel accountable and own what NGAS is relying on them to do. A signature sign-off must mean something. With this ownership mindset, people must believe they are empowered to call a halt to the execution of a process if it doesn’t look right or they don’t completely understand what they are being called upon to do.

The Webb IRB recommends that NGAS management maintain a failure-proof safety net to ensure mission success. This safety net encompasses testing, independent analysis, and inspection to ensure that the system design meets all of its requirements and any workmanship errors are detected and corrected.

The propulsion valve issue mentioned before and most recently the sunshield fasteners that came loose during acoustic testing of the Spacecraft Element are two examples of problems that were discovered later in the development phase than they should have been, with subsequent large impacts on cost and schedule for the remedial actions required. These experiences raise the possibility of other such issues remaining in the design at this stage. To attack this, the Webb IRB recommends [Goddard Space Flight Center (GSFC)] and NGAS conduct an audit to identify potential embedded problems still remaining in the observatory.

  • Robert G. Oler

    This project needs a halt, bring in someoutside experts and do a rethink…get new management…and see what can be salvaged from it…

  • jack dunster

    So what happened to NASA that the work force exhibits such a degree of incompetence? What happened – a generational attitude?

  • Andrew Tubbiolo

    None of these types of errors leads to these kinds of schedule slips, or budget overruns of thousands of millions of dollars. When I was doing spacecraft work I’ve encountered wiring errors, dropped CCD detectors, military U type sockets whos silk screens were mirrored, GSE cables with voltage and ground flipped, bad solder joints, you name it. Never did it result in a slip of launch date because of it. There’s another reason for delays like this, I smell design problems, failed qualification tests, or major instrumentation problems. The kinds of problems described here are run of the mill process errors, happen all the time, and are resolved quickly. For instance when I read “valve problems” due to the wrong solvent being used, that’s not going to cause a launch delay or go over budget to the tune of 100’s of millions of dollars or more. However a valve that can’t meet qualification criteria and is still under development can cause that kind of problem. I’m not saying it’s the valves, only giving an example of the kind of problem I suspect is going on here.

  • jack dunster

    ahh Mr Therealdmt.. Your great comment about a series of ineptness was spot on…. – and might go back even as far as the time of Mercury – the capsule that sank – and Apollo 1 – a grave example too. But surely, after one error there must be preventative steps taken… I do remember Challenger… and the committee set up under Neal Armstrong – yet that excluded the problem of icing which caused the lives of a crew. This speaks to me that there’s grave issues in the managerial attitudes of NASA. How did that saying go ‘one pioneering error – that a shame – one series of errors – that’s incompetence.

  • therealdmt

    The James Webb has been a real mess, no doubt. I was hoping that they would cancel it back when the first giant cost overrun was revealed.

    I definitely agree that one would have hoped for better management. In some ways, catching this stuff on the ground is actually a positive sign, but…

    Anyway, if it survives and actually launches, i’ll be pulling for it

  • therealdmt

    Hmm. Looks like there were some other comments by other posters that might have been removed too, in which case I may have unintentionally started a little flame war going or something. If that was the case, then i’m okay with removal and dousing the flames

  • envy

    The actual work mentioned is being done by Northrop, not NASA. NASA is managing the project but not running wires or tightening nuts and bolts.

  • mattmcc80

    That’s pretty much where I’m at now. Take the hardware away from Northrop and find another team to get it to the finish line. Before somebody trips over their shoelaces and breaks a mirror segment.

  • Robert G. Oler

    Yes or at least “think through” where the project is and where it is going…and use some folks outside of NASA and NG to do this. Both clearly are not capable of doing this effort

  • Robert G. Oler

    NASA”s role here is management…a attribute they have been losing steadily for decades now.

  • Cameron

    Cancel the program, send it to a museum and display it as an example of how not to do things. Put it next to the SLS EM-1 rocket.
    Then replace it with a CRS-style telescope program. NASA can get it’s money back on discounted rates for use of the telescopes (plural) for the first x number of years.

  • jack dunster

    so what you’re saying is that there is a culture of undermanagement/mismanagement and actual lack of thinking in the American professional/work forces in general… a failure in the school system?

  • Paul451

    OTOH, the report does say that NASA oversight often accepts Northrup’s word that something was done instead of actually verifying it. It seems to be a conspiracy of fools.

  • kmbog

    If another contractor were to be given the program they’d simply bring in probably in the neighborhood of 95% of the current work force.

    However as is so often the case in government procurement and programs of this magnitude there will probably be another contractor added but solely to watch the prime and sub-primes and address those critical issues.

  • Robert G. Oler