STS-93: Keeping Eileen on the Ground, Part 1

The summer of 1990 is remembered in shuttle circles as the hydrogen leak summer. We tried to launch Columbia and found that there was a leak in some of the plumbing carrying that volatile gas. The launch team scrambled into a series of hydrogen loading tests with various sensors to find the source. During the trouble shooting on OV-102, we tried to launch Discover and found leaks in OV-103 too. All summer we sat on the ground running test after test. There is a really good summary on the NASA lessons learned page: http://appel.nasa.gov/2008/01/01/the-summer-of-hydrogen/ This was the set up for our story about STS-93, the most interesting shuttle launch of them all.

Hydrogen is a powerful fuel for rockets, but it is hard to work with. The molecules are very small and can slip through the tiniest of openings; and exposed to any oxygen (in the air), well, everybody has seen the Hindenburg movie. Not good.

The aft compartment is the engine room of the space shuttle. It is surprisingly large and open on the inside – the volume is bigger than your bedroom but oddly shaped. And it is stuffed full of pipes of various sizes running every which way; large ones and small ones, some carrying gases and some carrying cryogenically cold fluids, hydrogen and oxygen lines, ammonia and Freon lines, hydrazine in two varieties and nitrogen tetroxide – a powerful oxidizer so deadly that it was said if you whiffed its fishy vapor in the air you were already dead. During main propellant loading and right up to engine start, dry nitrogen gas is pumped into the compartment to keep the possibility of fire as low as possible.

And to detect if we had a problem? We went through an evolution in sensor technology, but in 1999, inside the shuttle aft compartment was one (1) inlet to a hose to detect a hazardous gas leak. The detection equipment was actually located well away from the shuttle itself, buried deep in the mobile launch platform. A variety of a mass spectrometer searched for various gases and forwarded concentration information to a console operator in the firing room, three miles away. A sample of gas “collected” in the top of the aft compartment wended its way down through the tubing and plumbing maybe a hundred feet before hitting the analyzer. And the analyzer needed frequent calibration to ensure that it was operating correctly. This system could tell us, within a reasonable expectation, that we had a leak and what gas it was. But the detection system was practically useless to tell us where the source of the leak could be found. This system was slow, unwieldy, and the best we had at the time.

The launch commit criteria was carefully written to tell the operators exactly what concentrations at what times were allowable and which were not – for example, sometimes when External Tank loading was started the seals weren’t thermally stabilized and leaked a little until they chilled down. Later on, that same concentration of hydrogen free in the aft compartment wouldn’t be acceptable.

And at certain times, parts of the shuttle system were “committed” to launch – the firing room personnel were not longer required to monitor nor call a hold. But there was the big exception statement in the front of the LCC that required “senior console operators” to use their “best judgment” to call a hold to stop a hazardous situation, no matter how late in the countdown.

So it was a real surprise that during the first countdown attempt for STS-93, the senior haz gas operator called a cutoff – the technical term for a launch scrub that late – at T-8 seconds for high hydrogen gas concentration. There is a really good youtube video here: http://www.youtube.com/watch?v=T7vGqQUhciE – which has an erroneous title and writeup, but the video is good.

At T-10 seconds, the ground computers send the very last command the onboard system needs to fly: Go for Main Engine Start. With no other electronic word from the firing room computers, the shuttle launches itself. Right after that T-10 seconds command, the big sparklers at the base of the MLP towers fire off to ignite any small patches of free hydrogen that might have escaped from the main engines. At T-6.6 seconds, the orbiter computers command the main engines to start, staggered by about 120 milliseconds – center first, then left, and finally right. Shutting down the count at T-8 seconds is dicey. If the mains start there is a lot of free hydrogen left around and plenty of heat to ignite it. The ROFI sparklers were firing – those take 48 hours to replace – and if even the first engine cracks open one valve in the start sequence, it is a three week turnaround to remove and replace any SSME that started. So it was a surprise to get the call, especially after haz gas was “committed” for launch. It was one of those senior console operator things.

It is not technically a pad abort, no fire or smoke occurred – other than the sparklers – but the adrenaline is very high all around the program when it gets that close.

Safing was complete, the crew unstrapped and rode the elevator down to wait for the next attempt; 48 hours later at the earliest.

The KSC management was ready to pin a medal on the guy that called the hold. But the first story we got in Houston was that the sensor had just gone through an automatic calibration and the high concentration reading was a ghost.

I never heard the end of that story. Did Ozzie get a medal? Did we unnecessarily scrub a launch? Somebody needs to fill me in. All I remember is it set me up for a bad case of launch fever two days later.

That story next time.

About waynehale

Wayne Hale is retired from NASA after 32 years. In his career he was the Space Shuttle Program Manager or Deputy for 5 years, a Space Shuttle Flight Director for 40 missions, and is currently a consultant and full time grandpa. He is available for speaking engagements through Special Aerospace Services.
This entry was posted in Uncategorized. Bookmark the permalink.

15 Responses to STS-93: Keeping Eileen on the Ground, Part 1

  1. Walt Pinkston says:

    Left me on the edge of my seat!!!

  2. Valerie Neal says:

    Hi, Wayne. Can you add a y to Discover on line 5? Thanks! I’m eager to hear the rest of this story.

  3. markkirkman says:

    Hi Wayne,

    Am I a complete jerk if I (a former MPS instructor) correct/remind you that we started the SSMEs 3,2,1 (right was first and center last)??? ;)

    Thanks for continuing to post these stories, it is still very much appreciated by many of us!!!!

    Mark Kirkman

    • waynehale says:

      Thanks for reminding me. Too long since I took the SST class! I always got that wrong: and this one too: if I remember correctly you, for a command path failure you are supposed to turn off the AC to the engine controllers to shut the engine down, but I always moved my hand a row down and shut the helium regs. Not a good plan, right?

      • Mark Kirkman says:

        Yep, “Command Path Failure = AC/PB Shutdown”….and yes, it was very bad form to try and shutdown engines with helium switches or isolate helium leaks with the engine power switches. ;) If you recall we had those power switches surrounded by yellow tape, and also had installed little plastic yellow caps on the switch tips just for that reason. I actually use a photo slide of panel R2 as an example whenever I give discussions on cockpit layout and human factors.

        Mark

  4. cliff says:

    Continue please…

  5. David Sundström says:

    This is how Grant Cates, the Columbia flow manager at the time explained it: “The haz gas detection system, both the primary and backup systems have what we call a vacuum ion pump, it draws the hydrogen ions into the detector and on occasions those vacuum ion pumps can outgas or burp is what we call them, hydrogen ions back into the detector and then those ions will be subsequently be read again for a second time and it was that occurrence that happened to us at T-16 seconds.” “It does happen on occasion and typically when that happens they look suspicious in nature and we can detect at the time that the indication is probably a burp but the nature of this one was such that it mimicked a real hydrogen leak.”

  6. Beth says:

    Cliffhanging at the end of each chapter…just like the great read it is.

  7. Bob Walker says:

    Grant is working for SAIC at KSC. Ozzie is still around as well…. if you were in need of his perspective of what happened during and after the scrub. I believe one of the end results was the launch commit criteria was re-written to better account for the way the Haz Gas instrumentation behaves. I was in the firing room that day. It was intense moment or two.

  8. Colonel Vincent Alcazar, USAF says:

    This story is going to be documented within a larger, longer narrative form, say…like a book…right?

  9. Ozzie did get an award and a bottle of champagne from Astronaut Jerry Ross and yes it was an unnecessary scrub, but better safe than sorry.

    G. Thomas

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s