July 08, 2021 – Successful Completion of Multi-Day Test
NASA successfully completed a test of procedures that would be used to switch to backup hardware on Hubble in response to the payload computer problem. This switch could occur next week after further preparations and reviews.
July 06, 2021 – Multi-Day Test of Procedures Began Today on Hubble
A multi-day test of procedures that would be used to turn on backup hardware on the Hubble Space Telescope began today. Meanwhile, NASA continues to investigate the cause of the payload computer issue that began on June 13.
July 02, 2021 – NASA Completed Preparations to Test Procedures in Coming Week
NASA completed preparations to test procedures in the coming week that would be used to turn on Hubble backup hardware as a possible response to a payload computer issue. Investigation is ongoing into the cause of the problem.
June 30, 2021 – NASA Preparing for Procedures to Turn On Backup Hardware on the Hubble Space Telescope
NASA is taking additional steps to investigate the Hubble Space Telescope’s payload computer issue that began on June 13, suspending science observations. In parallel with the investigation, NASA is preparing and testing procedures to turn on backup hardware onboard the spacecraft. The telescope itself and science instruments remain healthy and in a safe configuration.
The source of the computer problem lies in the Science Instrument Command and Data Handling (SI C&DH) unit, where the payload computer resides. A few hardware pieces on the SI C&DH could be the culprit(s).
The team is currently scrutinizing the Command Unit/Science Data Formatter (CU/SDF), which sends and formats commands and data. They are also looking at a power regulator within the Power Control Unit, which is designed to ensure a steady voltage supply to the payload computer’s hardware. If one of these systems is determined to be the likely cause, the team must complete a more complicated operations procedure to switch to the backup units. This procedure would be more complex and riskier than those the team executed last week, which involved switching to the backup payload computer hardware and memory modules. To switch to the backup CU/SDF or power regulator, several other hardware boxes on the spacecraft must also be switched due to the way they are connected to the SI C&DH unit.
Over the next week or so, the team will review and update all of the operations procedures, commands and other related items necessary to perform the switch to backup hardware. They will then test their execution against a high-fidelity simulator.
The team performed a similar switch in 2008, which allowed Hubble to continue normal science operations after a CU/SDF module failed. A servicing mission in 2009 then replaced the entire SI C&DH unit, including the faulty CU/SDF module, with the SI C&DH unit currently in use.
Since that servicing mission, Hubble has taken over 600,000 additional observations to exceed 1.5 million during its lifetime. Those observations continue to change our understanding of the universe.
Launched in 1990, Hubble has been observing the universe for over 31 years. It has contributed to some of the most significant discoveries of our cosmos, including the accelerating expansion of the universe, the evolution of galaxies over time, and the first atmospheric studies of planets beyond our solar system. Read more about some of Hubble’s greatest scientific discoveries.
June 25, 2021 – NASA Completes Additional Tests to Diagnose Computer Problem on Hubble Space Telescope
NASA is continuing to diagnose a problem with the payload computer on the Hubble Space Telescope after completing another set of tests on June 23 and 24. The payload computer halted on June 13 and the spacecraft stopped collecting science data. The telescope itself and its science instruments remain in good health and are currently in a safe configuration.
The spacecraft has two payload computers, one of which serves as a backup, that are located on the Science Instrument and Command and Data Handling (SI C&DH) unit. There are various pieces of hardware which make up both payload computers, including but not limited to:
- a Central Processing Module (CPM), which processes the commands that coordinate and control the science instruments
- a Standard Interface (STINT), which bridges communications between the computer’s CPM and other components
- a communications bus, which contains lines that pass signals and data between hardware
- and one active memory module, which stores operational commands to the instruments. There are three additional modules which serve as backups.
Additional tests performed on June 23 and 24 included turning on the backup computer for the first time in space. The tests showed that numerous combinations of these hardware pieces from both the primary and backup payload computer all experienced the same error – commands to write into or read from memory were not successful.
Since it is highly unlikely that all individual hardware elements have a problem, the team is now looking at other hardware as the possible culprit, including the Command Unit/Science Data Formatter (CU/SDF), another module on the SI C&DH. The CU formats and sends commands and data to specific destinations, including the science instruments. The SDF formats the science data from the science instruments for transmission to the ground. The team is also looking at the power regulator to see if possibly the voltages being supplied to hardware are not what they should be. A power regulator ensures a steady constant voltage supply. If the voltage is out of limits, it could cause the problems observed.
Over the next week, the team will continue to assess hardware on the SI C&DH unit to identify if something else may be causing the problem. If the team determines the CU/SDF or the power regulator is the likely cause, they will recommend switching to the backup CU/SDF module and the backup power regulator.
Launched in 1990, Hubble has been observing the universe for over 31 years. It has contributed to some of the most significant discoveries of our cosmos, including the accelerating expansion of the universe, the evolution of galaxies over time, and the first atmospheric studies of planets beyond our solar system. Read more about some of Hubble’s key scientific contributions.
June 22, 2021 – Testing Underway to Identify Issue and Restore Payload Computer on NASA’s Hubble Space Telescope
NASA continues to work to resolve a problem with the Hubble Space Telescope payload computer that halted on June 13. After performing tests on several of the computer’s memory modules, the results indicate that a different piece of computer hardware may have caused the problem, with the memory errors being only a symptom. The operations team is investigating whether the Standard Interface (STINT) hardware, which bridges communications between the computer’s Central Processing Module (CPM) and other components, or the CPM itself is responsible for the issue. The team is currently designing tests that will be run in the next few days to attempt to further isolate the problem and identify a potential solution.
This step is important for determining what hardware is still working properly for future reference. If the problem with the payload computer can’t be fixed, the operations team will be prepared to switch to the STINT and CPM hardware onboard the backup payload computer. The team has conducted ground tests and operations procedure reviews to verify all the commanding required to perform that switch on the spacecraft.
If the backup payload computer’s CPM and STINT hardware is turned on, several days will be required to assess the computer performance and restore normal science operations. The backup computer has not been powered on since its installation in 2009; however, it was thoroughly tested on the ground prior to installation on the spacecraft.
The payload computer is a NASA Standard Spacecraft Computer-1 (NSSC-1) system built in the 1980s that is located on the Science Instrument Command and Data Handling (SI C&DH) unit. After 18 years on orbit, the original SI C&DH experienced a failure in 2008 that delayed the final servicing mission to Hubble while a replacement was prepared for flight. In May 2009, STS-125 was launched and the astronauts installed the existing unit. The replacement contains original hardware from the 1980s with four independent 64K memory modules of Complementary Metal-Oxide Semiconductor (CMOS) memory. Only one memory module is used operationally, with the other three serving as backups. All four modules can be used and accessed from either of the redundant payload computers.
Launched in 1990, with more than 30 years of operations, Hubble has made observations that have captured imaginations worldwide and deepened our knowledge of the cosmos.
For more information about the Science Instrument Command and Data Handling unit view the following PDF: Science Instrument Command and Data Handling unit excerpt
June 18, 2021 – Operations Continue to Restore Payload Computer on NASA’s Hubble Space Telescope
NASA continues to work on resolving an issue with the payload computer on the Hubble Space Telescope. The operations team will be running tests and collecting more information on the system to further isolate the problem. The science instruments will remain in a safe mode state until the issue is resolved. The telescope itself and science instruments remain in good health.
The computer halted on Sunday, June 13. An attempt to restart the computer failed on Monday, June 14. Initial indications pointed to a degrading computer memory module as the source of the computer halt. When the operations team attempted to switch to a back-up memory module, however, the command to initiate the backup module failed to complete. Another attempt was conducted on both modules Thursday evening to obtain more diagnostic information while again trying to bring those memory modules online. However, those attempts were not successful.
The payload computer is a NASA Standard Spacecraft Computer-1 (NSSC-1) system built in the 1980s that is located on the Science Instrument Command and Data Handling unit. The computer’s purpose is to control and coordinate the science instruments and monitor them for health and safety purposes. It is fully redundant in that a second computer, along with its associated hardware, exists on orbit that can be switched over to in the event of a problem. Both computers can access and use any of four independent memory modules, which each contain 64K of Complementary Metal-Oxide Semiconductor (CMOS) memory. The payload computer uses only one memory module operationally at a time, with the other three serving as backups.
Launched in 1990, Hubble has contributed greatly to our understanding of the universe over the past 30 years.
For more information about Hubble, visit: www.nasa.gov/hubble
June 16, 2021 – NASA is working to resolve an issue with the payload computer on the Hubble Space Telescope
NASA is working to resolve an issue with the payload computer on the Hubble Space Telescope. The computer halted on Sunday, June 13, shortly after 4 p.m. EDT. After analyzing the data, the Hubble operations team is investigating whether a degrading memory module led to the computer halt. The team is preparing to switch to one of several backup modules on Wednesday, June 16. The computer will then be allowed to run for approximately one day to verify that the problem has been solved. The team would then restart all science instruments and return the telescope to normal science operations.
The purpose of the payload computer is to control and coordinate the science instruments onboard the spacecraft. After the halt occurred on Sunday, the main computer stopped receiving a “keep-alive” signal, which is a standard handshake between the payload and main spacecraft computers to indicate all is well. The main computer then automatically placed all science instruments in a safe mode configuration. Control center personnel at NASA’s Goddard Space Flight Center in Greenbelt, Maryland restarted the payload computer on Monday, June 14, but it soon experienced the same problem.
The payload computer is a NASA Standard Spacecraft Computer-1 (NSSC-1) system built in the 1980s. It is part of the Science Instrument Command and Data Handling module, which was replaced during the last astronaut servicing mission in 2009. The module has various levels of redundancy which can be switched on to serve as the primary system when necessary.
For more information about Hubble, visit: www.nasa.gov/hubble