“The Space Shuttle is an experimental vehicle with an operational mission” – NASA Deputy Associate Administrator Michael Kostelnik, 2004
The Space Shuttle system was under development for 13 years and then actually flew in space for over 30 years. Its main engines were the most tested in history with over one million seconds of firing time in ground test stands. Equipment testing and inspections continued unabated from the early development days right up until the final flight, and even sometimes beyond. Tons of documents contained the data from millions of qualification and certification tests for every system, each subsystem component, every piece part, every bolt, every nut, every electronic chip that ever flew as part of the shuttle. Computer simulations of every conceivable operation of the shuttle system consumed uncounted terabytes of memory. Computational Fluid Dynamics analysis were run with the highest fidelity, finite element structural models were tested with every possible loading conditions, Monte Carlo simulations for the navigation and flight control operations combined every possible variation in systems performance, environmental variation, trajectory offset. If there was ever a mature space system, the space shuttle was it.
So, after the turn of the millennium, when individuals or groups would suggest that perhaps we did not know enough about how the shuttle worked, you can understand why senior management brushed concerns aside with the rejoinder: ‘after all, the shuttle is a mature system, we know how it operates.’
Seen from another angle the view might be different. All those parts were tested to the environment (loads, pressures, temperatures, etc) that we thought they might see in actual use. Mostly we got that right; sometimes we found out that we didn’t really know. And the real, total system, every time it flew, it flew right down the center of the envelope. Because of cost, risk, expense, focus on mission objectives, whatever; we avoided those tests that might really stress the system. Mostly those tests were flown in simulations, with computers, based on assumptions, and models; not the real world.
And sometimes the results of real flights, right down the middle of the envelope, had puzzling results. Because they didn’t fit with the cultural view that the shuttle was a well understood, mature system, sometimes not enough attention was paid to those pesky anomalies. So we just made a convoluted definition of what was an anomaly and ruled out some events as if they had never happened.
Lesson point: don’t let the great mass of information that says that everything is going well blind you to the uncomfortable evidence that something isn’t right. Really high reliability organizations are obsessed with the possibility of failure.
Space flight is not like commercial aviation, but consider this sticky story. Commercial airplanes, being certified by the FAA to gain an airworthiness certificate, fly not just 4 times, or even just 135 times, but thousands of times before being declared “operational.” Not that the FAA doesn’t appreciate analysis and simulation, but those pesky regulators require that the airplane really fly: the whole airplane, in the real environment, flying every test, every time. And not only that, the FAA requires testing what the engineers call ‘the corners of the envelope.’ Conditions where things might not go well; like shutting down an engine during takeoff. Conditions like landing at maximum weight and maximum speed on the shortest allowable runway and see if the brakes can really stop the plane. All of these tests and many more are the standard stuff of aircraft certification; virtually unheard of in space systems testing; not with the real flight vehicle, not in the real flight environment.
But those of us in the space shuttle program, though we knew about it, rarely talked about our immaturity. So the conventional wisdom grew that the Space Shuttle was a “mature” system with few surprises left to learn. The new hires and the marginally informed were easily won over. The old guys, who had lived through the Apollo 1 fire and Apollo 13, they were not fooled. And those of us in the middle remembered Challenger and were vaguely, inarticulately uncomfortable. We remembered Challenger, but with the wrong lens: a bad manager made one bad decision, that was what caused Challenger. But it really wasn’t that simple. It was a whole group of people who were blind because their culture taught them to be blind.
So start with a culture where the conventional wisdom confirms that the operations are mature, well understood, with few surprises left to encounter. Mix in a little schedule pressure. Compress it all with a huge amount of financial squeeze. What do you get?
Who did the Monte Carlo run on that simulation? And where are the results documented?
A lot of people think NASA is risk averse; or even that our country is risk averse. I think the opposite is true; we are willing to take great risks. It’s just that sometimes we are not very smart about taking risks.
The number one lesson in taking risks? Don’t fool yourself.
Wayne Hale's Blog 
Oh so true. Yes take a calculated risk. If it goes pear shaped, stand back put it right ang go for it again. You never learn by getting it right all the time.
Wayne, I do not believe I have ever heard it explained as succinctly or more clearly. As much as was known there were always those unknown unknowns that would pop up once in a while that defied understanding. Excellent posting. Thank you.
Excellent, Wayne. With 135 flights, not counting Enterprise drop tests, the Shuttle was still an experimental flying machine. I think the program was burdened by the sales pitch made for it early on as an inexpensive and reliable and frequent space transportation system. Maybe this was the only way to sell it to a disinterested Congress, but it was a chronic blight on the true understanding of the program.
Good reminder of the Approach and Landing tests that Enterprise and the Shuttle Carrier Aircraft performed. I’m thinking that we had to do more than a 135 flights to get the SCA approved by the FAA
“The number one lesson in taking risks? Don’t fool yourself.”
I like to say, “Lie to whoever you want, but never lie to yourself”
Your last two paragraphs ring all too true for me, Wayne. I’m often perplexed at the take our fellow Americans have on everyday risk. I know many folks out there who are terrified at the prospect of flying commercially, even on rare occasions under high motivation. Yet the same individuals have no problem using public highways on a daily basis, where the risk of injury and death is far greater.
Wayne, this is your best yet. If people read just one, this should be it. It captures for me my lasting view beginning to end, that explains how we collectively fooled ourselves. Which is not to take from the majestic aspect of the accomplishment of the Shuttle that it was.
But lets be blunt – our challenge in pursuing greatness is not accomplishment in spite of various imposed (budgetary, initial compromise, managerial/ops) or accumulated (presumption of knowledge, unexamined assumptions, lack of IR&D to explore passed over options / inexplicable behavoir, …), but in being encompassing these as part of the selfsame greatness.
The Shuttle was viewed too narrowly as a means to an end. Budget/management/other collapsed around it, self limiting the experience/”greatness”. From the beginning. To the end.
Many knew it. But how do you raise the issue without seeming to oppose its greatness. How to keep a civil dialog longitudinaly … without “farting in church”.
What I liked about Columbia RTF was a belief in follow-up … that should have been there from before TAOS decision. And stayed. Results from STS-1 warned us of this.
Excellent article Wayne. Group think at it’s finest.
I tend to think of group think as something that happens periodically in a short term situation. I guess you could apply that term to the culture of the shuttle program but it was a long term (more than a decade) phenomenon.
Wayne I really enjoy reading your thoughts and recollections. Does anyone remember that the Shuttle was declared “Operational” after only four flights. No wonder there was a diverse culture within the Program with regard to flying a space ship versus what someone once called an airplane with tiles.
NASA is willing to put people on the second flight of a new vehicle. Compare this with SpaceX that would have dozens of flights before risking people (yes, some systems having less flight heritage then others.)
Second flight? We put real live people on the very first flight of the Space Shuttle.
And why are you dragging in a commercial for SpaceX?
Wayne,
The concept of a “mature” space transportation system at this point in history is somewhat presumptuous at best. If, by the use of the word “mature” we presume to know and understand every possible risk, we leave the path to complacency unobstructed.
History shows us that when we presume something cannot happen then it will happen. I believe that the above is one of Murphy’s Laws.
A more recent example can be described by one word: Fukushima.
Wayne, could we have been more agressive with the program had a robust crew escape capability been designed in from the beginning? What about an unmanned Orbiter option?
More agressive? Surely you jest. Crew escape should have been designed in from the beginning just to do the things we actually did. And I don’t mean a couple of ejection seats that would only have helped in the last stages of the landing.
Unmanned orbiter? What would be the point? You can launch a lot of stuff on an expendible rocket but you don’t have the people there to cope with it when it doesn’t work right.
Well, I suppose we could ask the USAF what it is they’re doing with the X-37B to get an idea of what point there is an unmanned orbiter, though I do not think at present they are likely to answer except in vague terms as the missions are classified. Certainly, though, it cannot do the same things it can if it has people in it, and Shuttle was meant from the start to do those things that need people. The thinks I suspect they are doing up there do not need people — and also do not need something as big as Shuttle. Different birds for different jobs.