Earlier this week, NASA published a Selection Statement explaining how and why it awarded $1.1 billion in commercial crew contracts to Boeing, Sierra Nevada Corporation and SpaceX last month. The 13-page document, written by William Gerstenmaier, NASA’s associate administrator for Human Exploration and Operations, lays out how the NASA rated the Commercial Crew Integrated Capability (CCiCAP) proposals submitted by these three companies and ATK, which did not receive funding.
A summary of the document with the ratings and rationale is below.
NASA evaluated proposals from ATK, Boeing, Sierra Nevada Corporation, and SpaceX. Proposals from three other companies — American Aerospace, Space Operations and Spacedesign — were not evaluated because they failed to meet basic proposal requirements.
NASA’s Participant Evaluation Panel (PEP) evaluated each of the four proposals according to the following criteria. There are Effectiveness and Confidence levels associated with each proposal’s technical approach and business information (plan).
The PEP conducted initial evaluations in which they rated each proposal. Final evaluations with due diligence were then conducted on all four proposals, resulting in higher ratings in most cases. The chart below shows the final ratings with contract award amounts.
FINAL EVALUATION AFTER DUE DILIGENCE
|Sierra Nevada Corporation||Dream Chaser/
Below are key excerpts from the Selection Statement. The first section includes Gerstenmaier’s comments on the individual proposals. The second section examines how he decided to allocate the funding among the three companies selected. My bottom line summaries are in italics.
EVALUATION BY COMPANY
The Bottom Line: Strong business case and financial commitment, but insufficient technical information and higher risks result in no joy in Utah.
“ATK’s proposal had a strong business case that leveraged Liberty’s characteristics and capabilities, such as the ability to support a wide variety of missions beyond crew transport, including non-NASA customers, through multiple vehicle configurations for crew and cargo, and the proposal showed a significant financial investment by the ATK team as compared to the requested NASA contribution, which gave me confidence in the company’s commitment to this activity.
“However, I had serious some significant concerns about the lack of detail in some areas of ATK’s technical approach that outweighed its strengths in the Business information section. In reviewing the PEP findings, ATK was rated the lowest of the four proposals in Level of Effectiveness for Technical Approach, with a rating of WHITE (Moderate). The PEP also identified more technical weaknesses (3) for ATK than for the other proposals. The first two of these three weaknesses would have a significant effect on the ability of ATK to meet the goals of the Announcement. The PEP also had a Technical Level of Confidence rating of LOW, which was the lowest of all proposals….
“Basically the proposal lacked enough detail to determine if a safe crew transportation system could be developed in a timely and cost effective manner out of the heritage components ATK selected for this concept.”
Boeing — $460 Million
The Bottom Line: This company knows how to build spacecraft, but its failure to put more skin in the game could cause problems down the line.
“A significant factor to me in Boeing’s proposal was the comprehensive approach Boeing has to its overall integrated vehicle development plan which includes an incremental approach to the detailed design and development effort, including subsystem-level Critical Design Reviews (CDRs) culminating in its integrated CDR. This approach did a good job of tying the milestones to completion of certification and qualification activities and will allow for greater review and understanding of technical products prior to advancing to later development milestones…
“Also, it’s proposed use of heritage systems at the subsystem levels provides confidence that design and development can be executed per the proposed schedule….
“It is important to note that Boeing’s proposal has a single weakness in the Business section after due diligence for not achieving the significant financial investment strategic goal laid out in the Announcement. While this is only one of 13 Goals, I did consider it. However, Boeing met all the other goals and had a strong technical design; therefore, I did not find the lack of significant corporate financial commitment to be a major discriminator in my assessment of the Boeing proposal….
“[The] proposed corporate investment during the CCiCAP period does not provide sufficient industry investment and there is increased risk of having insufficient funding in the base period….”
SpaceX — $440 Million
The Bottom Line: Best business plan of the bunch, but more information needed on how to get from cargo to crew.
“I find SpaceX’s ability to leverage its existing Dragon/Falcon cargo system with incremental targeted design upgrades reduces the overall scope of the development effort going forward. Furthermore, their ground systems and mission control capability will be demonstrated several times on cargo mission prior to making the step to crewed missions, which is an advantage….
“Leveraging the CTS off of the current cargo systems offers SpaceX a strong technical advantage. However there is a technical weakness identified by the PEP in this area. The PEP was concerned about the lack of sufficient detail provided to show how SpaceX will mature the integrated CTS from the current cargo configuration to a human configuration. I share that concern, but temper the concern with the advantages of having flown the basic cargo system. Flight experience with the cargo version will show areas of the design that need additional work….
“SpaceX’s Business Information received the highest allowable Effectiveness rating of BLUE (Very High) with the Highest allowable Confidence rating of High. SpaceX’s ability to utilize significant hardware elements of its existing cargo system, infrastructure and processes is consistent with it having the lowest overall development cost of all the proposals. The proposal sets out a credible business plan on how SpaceX will capture different markets to LEO and ISS and how it will utilize existing revenue streams associated with its current manifest to fund the CCiCAP effort.”
The Bottom Line: A winged lifting body has unique capabilities and unique challenges; the latter make this a higher-risk (and lower-rated) proposal.
Technical approach “does not adequately address the risk associated with spacecraft Thermal Protection System (TPS damage).”
“SNC has many more technologies that need to be matured in order to reach a CDR and its winged lifting body design presents some unique challenges not found in a capsule design. This technology development and the rest of its development activities within the CCiCAP base period makes SNC’s proposal a higher risk choice.
“However, there are some distinct advantages to keeping SNC and its winged vehicle approach in the portfolio. The use of a winged lifting body offers lower entry and landing g-forces, which can be easier on humans and can enable more scientific payloads that require a smoother landing to be brought back from space, cross range capability (ability to deorbit on orbits not directly aligned with the runway), and ruway landings which enable quick crew egress. These significant technical advantages could result in a larger customer base for SNC than a capsule design and more capability for users…
“SNC had significant risks because of design complexity that could impact the business approach. This resulted in a Medium Confidence rating for SNC’s business approach.”
The Bottom Line: Boeing has the best technical chops, SpaceX can get to a crewed flight fastest and cheapest, and SNC still has a bit of work to do.
“In reviewing the remaining three proposals, while all three received a Very High Level of Effectiveness rating for the Technical Approach, it seemed clear to me that SpaceX and Boeing had the stronger technical proposals. This is evidenced by the Medium technical Confidence rating for SNC.
Even though SpaceX also received a Medium technical Confidence rating, I do not consider these technical Confidence ratings of Medium for SNC and SpaceX as equivalent. The SNC Confidence rating is driven by several factors; complexity of heat shield design, complexity in abort conditions, controlling weight of the design, and ability to bring green propellents on line in a cost effective and timely manner. Whereas the SpaceX design is driven predominantly by a single consideration: design changes from the current cargo vehicle to a crew vehicle and human rating. Boeing had a higher technical Confidence rating….
“This portfolio provides for a diversity of spacecraft designs (capsule and winged lifting body) as well as capturing the proposal that provides the earliest crew demonstration flight under a credible schedule at the lowest development cost (SpaceX) with the proposal that represents the highest Level of Effectiveness and Confidence ratings on the technical approach (Boeing). Carrying a third company in the portfolio also adds robustness, if one of the three were to run into significant technical or financial trouble. Carrying a third company would keep competition even if one company needed to drop out….
“SNC has the most significant amount of risk reduction and technology development work to do before reaching CDR, and I would like to see what kind of progress SNC can make on increasing the maturity of of some of its key technologies before providing them with additional funding. For this reason, I have decided that SNC will receive a significantly reduced award. In order to have sufficient funding, further reductions were also needed in Boeing’s and SpaceX’s original proposals. The reductions were moderate and determined to not adversely affect the proposals.”
You can download the full Selection Statement here.