STS-121: The Hardest Launch – part 1

“The hardest part of any launch is getting the first foot off the ground” – Tommy Holloway, Space Shuttle Program Manager

I was doing some online research recently and ran across this document which I had not seen in over a decade:  https://www.nasa.gov/pdf/150505main_121_CoFR.pdf

The picture shows Associate Administrator Bill Gerstenmaier signing the STS-121 CoFR document following the Flight Readiness Review.  Over his shoulder is NASA Administrator Mike Griffin who played a pivotal role at the FRR.  He is deep in discussion with Chief of NASA Safety, Bryan O’Connor (back to the camera)

A good, gray, boring government document comprising the signature pages from STS-121 Flight Readiness Review.  Signatures showing that all the senior management agreed to the launch of STS-121, except, if you go to the last couple of pages, not all of them agreed!  This is the only case I know of where such dissent happened regarding a shuttle launch.  The entire story of STS-121 has multiple lessons that may apply today as well.  Buckle in and I will tell you the story – but remember it is my perspective and I’m probably biased and sometimes forgetful.

It has been five years since I wrote about how we nearly lost Discovery on the return to flight after Columbia:  https://waynehale.wordpress.com/2012/04/18/how-we-nearly-lost-discovery/

After two and a half years of trying to correct the problem that caused the loss of Columbia, the launch of STS-114 proved we had failed.  Before we could even start to make corrections, Hurricane Katrina smashed into New Orleans and complicated everything.

Setting aside the serious issues of the PAL ramp and recovering from Katrina, there were at least three other major problems associated with returning the shuttle to flight status (again!) which were not completely resolved at the time we were ready to launch STS-121:  foam loss from the ‘ice/frost ramps’; thermal scan indications of manufacturing anomalies in the RCC leading edge panels; and the nagging issue of intermittent failures in the engine cut off sensor system.  I plan to write a separate background post on each of these issues and then a summary of the drama and decision that occurred in mid-June 2006.

Returning again to the major anomaly on STS-114, the loss of a large section of foam insulation from the ‘protuberance air load ramp’ (called the PAL ramp) was a surprise.  The Shuttle senior leadership team had considered and discussed removing that 17 pound sprayed on foam barrier which was the PAL ramp, but decided against removing it.  John Muratore was one of the leading proponents of removing the PAL ramp and we probably should have listened to him.  The data in 2004/5 showed historic foam insulation loss from the PAL ramp was extremely rare and limited, and the defect/divot foam loss mechanism did not apply to the PAL ramp.  The major error in the logic, found after STS-114, was that the defect/divot foam loss mechanism was not the real problem; differential coefficient of thermal expansion between foam layers was the real problem.  Go back to the earlier Discovery post for more details.  After STS-114 it was absolutely obvious that removal of the PAL ramp had to be done.

The PAL ramp was a late addition to the External Tank prior to the first flight of the shuttle back in 1981.  Aerodynamic modeling was not very sophisticated in those days and there was an indication that the ‘protuberances’ might be ripped off the outside of the tank in some extreme cases.  The ‘protuberances’ – shown in the attached picture – consisted of the 17 inch diameter pipe carrying the liquid oxygen from the front part of the ET, a rectangular cable tray that carried critical instrumentation signals, and two smaller metal pipes carrying gaseous hydrogen and gaseous oxygen (in different pipes!) heated by the engines in the aft end of the Orbiter, to maintain pressure in the External tank as the LH2 and LO2 were pumped out to feed the engines.  All of these ‘protuberances’ were attached with metal fittings to the aluminum skin of the ET.  The attachment fittings were chilled by the cold fluids inside so those fittings were insulated with foam; these insulated foam fitting were called ‘ice/frost ramps’ since they had to prevent the formation of frost or ice and had to be ramp shaped due to the aerodynamics.

Nothing is easy.

To remove the PAL ramp and eliminate the danger of chunks of foam coming off and damaging the orbiter heat shield, we had to first prove that the aerodynamics concerns identified back in 1981 did not exist.  This required an extensive wind tunnel test campaign to measure forces followed by analysis using computational fluid dynamics models to check the cases the wind tunnel testing could not do.  Attached is a picture from the Glenn Research Center wind tunnel test article showing the sub scale protuberances that were tested there.

Building models, scheduling wind tunnel time, checking the results all took months.  Most of the work wrapped up in May with a few loose ends.  Removing the PAL ramp was proven to be safe; but in the mean time there were other problems.  More on that next time.