Lessons not to be forgotten

This month contains NASA’s Day of Remembrance as it does each year.  Each year the events that need to be remembered draw more deeply into the past. 

This year is extraordinary because it marks the twentieth since we lost Columbia and her brave crew.

There are many who have come into the human spaceflight community following that accident.  Some are too young to remember the events at all.  Soon, joining the ranks of those who propose to send frail humans into the cosmos, will be ones who were not even born then. 

These fresh faces, building the future, must understand not only what happened but why it happened and how to prevent such ruinous tragedy from happening again.

Ten years ago, at the encouragement of an old friend, Lisa Martignetti (who had her own role to play), I wrote a series of essays encompassing my memories, observations, and thoughts about the tragedy; what came before, how it transpired, and the way ahead.

It is my proposal to start reposting those remembrances, one a day, starting tomorrow January 15 and continuing to the conclusion on February 1.  This pace should allow for some thoughtful reflection each day.

I hope these essays will lead the readers to find a better understanding of the complex nature of failure and – much more importantly – how to avoid making the same mistakes.

I hasten to add that I have not modified these essays from their original content.  My memory grows less precise and I do not want to alter what I had written lest I introduce more error.  There are certainly areas where other eye witnesses remember differently, that is the fact of human experience: we all see the same events from different vantage points and perspectives.  I do not intent to defend or amend what I have written as it is the best my memory can produce. 

Shortly after the CAIB report published when we were trying to learn better ways, I invited a series of speakers to interact with the Shuttle leadership team.  First was Dr. Charles Perrow, a Yale sociologist, who had written a well-respected book entitled “Normal Accidents:  Living with High-Risk Technologies”.  His basic thesis states that complex, tightly coupled technical systems will always fail – that is ‘normal’.  We who had just lived through tragedy were looking for ways to prevent failures in the future.  Dr. Perrow was relentless; he gave us no hope.  He stared us right in the face and said we were attempting the impossible.  We should inevitably expect such a failure to happen again and nothing we could do would prevent it.  Human error would creep in and defeat any protective system we might devise.

We could not accept that.  It was the most disheartening talk ever.  We could not wait to get him to leave so we could make plans to prove him wrong. 

To some extent we succeeded; the remaining shuttle flights were all safe and successful. 

So, is that the end of the discussion?

Lately I have begun to wonder if he was right.  We are building and flying large complex, tightly coupled space systems with younger and less experienced workers.  How can they learn the lesson?  The only lesson I know to keep the wolf at bay, even for a short while, is extreme vigilance from everybody involved.

Please read and reflect on the essays to follow in the hope you can prove Dr. Perrow wrong. 

The View from Mt. Nebo

When I was a child, 5 or 6 years old, I had a recurring dream that I could fly.  Not fly an airplane but fly like a bird.  Although better than any bird because I could traverse continents in minutes and visit strange and exotic places.  All I had to do was stretch out my arms and point my toes and away I went.  There were not daydreams but recurring dreams that I would recall when awake.   Like all dreams they were kaleidoscopic but vivid.  I could recall them in detail when I awoke.  In fact, I can remember some of them even now, decades later.  Friends, schoolmates, or family rarely showed up; mostly they were about the exhilaration of flying and traveling far from home.

 Now I was never foolish enough, in my waking state, to believe I could really fly; never tried to jump off the roof of the house.  But the memory remains.  Possibly many other young children have this same dream.  Maybe some child psychologist can analyze this, but I can’t.  I can only say that they meant freedom and adventure and a future of unlimited possibilities. 

Over time the dreams changed to other things and now my adult dreams are chaotic, symbolic, and quite literally undecipherable, on those occasions where I remember them.

I grew up with the space age.  Three years old when Sputnik launched, the adventure captured my imagination from earliest childhood.  I ravenously consumed every book, article, and news story about the burgeoning space race:  Ranger, Surveyor, Mercury, Gemini, and finally Apollo were the background tapestry of my childhood and my youth.  The moon missions followed by Skylab and ASTP ran into my college years. 

In the midst of this I was captured by the classic science fiction of the era:  written works from Jules Verne, Robert Heinlein, Isaac Asimov, and more.  My generation were all destined for a glorious future traipsing about the universe. Or blowing ourselves to bits in a nuclear holocaust with not much anything in-between.  My impressionable mind was influenced by the classics of science fiction in film and video:  Star Trek, Lost in Space, Space 1999, Destination Moon, The Forbidden Planet, a host of B movies mostly about monsters in space, and the like. Star Trek with its inherent optimism showed a better future to aspire to. (Star Wars came later, just after I married my wife).  I still consider ‘The Moon is a Harsh Mistress’ to be my favorite book and the blueprint for what should have followed. 

After the moon landings while I was still a teenager, I felt bitter disappointment at the decisions not to press on to Mars.  Confusion and complete non-comprehension of any reason to stop.  It all seemed so obvious:  back to the Moon to stay, building bases and colonies, and then on to Mars, the moons of the outer planets, and someday, someday, someday, to the stars. Just waiting for Zefram Cockrane.

Thus passed what we have come to call the Apollo generation:  especially those who worked in the space program during the heady days of the 1950’s all the way through the moon landing and Skylab to ASTP in 1975.

Filled with joy at being selected to work in NASA’s Mission Control just before the first launch of the Space Shuttle Program.  The Space Shuttle was just being completed and a flying taxi to low earth orbit was the obvious next step in the plan, right from the Willy Ley/Werner Von Braun TV specials on The Wonderful World of Disney.   In my early adulthood, it seemed certain that we would do this ‘shuttle thing’ for a few years, followed by building the space station.  That orbiting base would certainly be the assembly point to kick out to lunar bases, on to Mars and then all the rest of the solar system.  I was happy in the prospect of a career spent helping humanity spread throughout our solar system.

Needless to say, it didn’t quite work out that way.

I wonder how Admiral Kirk – or Admiral Picard – would have dealt with budget cuts proposed by the Federation Senate – financial constraints based on the premise that we need to solve problems on earth before we take off for the stars.  Hollywood never wrote that episode for TV.

So, my career was spent in doing my best to cart humans back and forth to low earth orbit, including building a space station that was never going to be the jumping off point for the moon.  I don’t regret it, we did good work, had a couple of really bad days, and hopefully laid the foundation for what is to come.   

We carried the torch, keeping human spaceflight alive to see better days. 

Not yet the Artemis generation, those days to come.  Those of us who labored in the 1980s to 2011 must be called the Shuttle/Station generation. 

Now I approaching my dotage, mostly retired, I serve only by giving Dutch uncle advice to the new generation.  Not exactly contributing in the way I expected.  But watching and hoping that they will succeed where my generation did not.  The generation that is, that came between Apollo and Artemis.  The forgotten generation. 

Watching with surprisingly mixed emotions as Artemis takes flight and shows the promise of success, after all these years.  The Artemis generation now in its initial days. 

I feel empathy for the folks still down in the ISS program; every day doing the science, making sure the station is supplied and equipped, planning EVAs for upgrades, making sure the international team holds together.  It is a 24/7/365 job that has just gotten eclipsed by the Artemis mission.  Keep carrying the torch you station guys.

Finally, there are mixed emotions: Joy and Pride and Gladness that the first Artemis mission has taken place and gone so well. Disappointment and sadness – and just a little bit of anger, too – that it has taken so long.  Jealousy of the folks in Mission Control and the engineering support rooms where I always dreamed I would be. 

Forlorn at the amnesia that has developed over what we accomplished with the Space Shuttle for all those decades.  Did we not set the foundations for today’s generation to succeed?

I think I will concentrate on the pride and joy that we are moving forward after all.

Justification, if you will, for all those years of service in low earth orbit. 

That is my view from Mt Nebo. 

Wars and Rumors of Wars

My Great-Grandmother died when I was in grade school.  By then I had already gotten a bookish reputation, so she made sure I inherited two books from her library.  The first was a religious book, ‘The Manliness of Christ’ published in about 1908.  It represents a very 19^th century point of view. 

The other slender volume is “Notable Events of the Nineteenth Century’ published in 1896.  This is a fascinating read. I was particularly interested in the section entitled ‘Progress in Discovery and Invention’ where there are articles such as “The First Steamboat”, “The New Light”, “The Machine that Talks Back”, and “The Unknown Ray”.  Science is represented by “The Century of the Asteroids” pointing out that the first asteroid, Ceres, was discovered and named in 1801, “The Evolution of the Telescope” followed by an article entitled “What the Worlds are Made of” which is, by the way, completely wrong as we know it today. Perhaps more apropos for our time is the article entitled “The New Inoculation”, followed by “Koch’s Battle with the Invisible Enemy”. 

A large part of the “Notable Events” book is taken up by “The Great Battles”.  Here is a list:  Trafalgar, Austerlitz, Waterloo, Sebastopol, Sadowa, Mexico City, Vicksburg, Gettysburg, Appomattox, Sedan, Metz.  Some of those are familiar but some I confess there are several I had never heard of before. 

Reminds me of a poem by Carl Sandburg that has this stanza:

“Let it be a series of memorials to the Four Horsemen, to Napoleon, Carl the Twelfth, Caesar, Alexander the Great, Hannibal and Hasdrubal, and all who have rode in blood up to the bridles of the horses calling, Hurrah for the next who dies, He was pretty good, but he didn’t last long.”

War and bloodshed are the curse of humanity.  We need to pray according to the old hymn “Father, stop thy children’s warring madness.” 

At the end of the 19^th century the great thinkers of the time believed that mankind was becoming so perfected that war would never again occur. We know how that turned out in the twentieth century.

The ancient Greek philosopher Plotinus postulated that evil was the absence of good, much as darkness is the absence of light.  He was wrong.  Evil is very real and active.

My inspiration for decades has been Archibald McLeish’s essay “Riders on the Earth Together, Brothers in the Eternal Cold” which was written just after Apollo 8 returned the first glorious pictures of the earth from a lunar perspective.  He concluded: “To see the earth as it truly is, small and blue and beautiful in that eternal silence where it floats, is to see ourselves as riders on the earth together, brothers on that bright loveliness in the eternal cold — brothers who know now they are truly brothers.” 

It has been my hope that the peaceful cooperative exploration of space would provide an outlet for humanities energies that would be peaceful and productive.  And indeed, onboard the International Space Station there is only talk of cooperation and teamwork.   Maybe there is still hope.

That hope is likely insufficient.  Evil and warfare does exist in the world, it is conspicuously present in the world today.   It is up to the people of goodwill everywhere to blot out that evil and stop our warring madness. 

May the peacemaker’s time be at hand. 

I will leave you with this thought from James Russell Lowell “The Present Crisis” 1845:

“Once to every man and nation, comes the moment to decide,
In the strife of truth with falsehood, for the good or evil side;
Some great cause, some great decision, offering each the bloom or blight,
And the choice goes by forever, ’twixt that darkness and that light.”

Flying New Spacecraft

Watching the commissioning of the James Webb Space Telescope, the control team took a couple of days to just monitor how the new space vehicle was performing.  That the spacecraft was operating a tad different than pre-mission predictions is not really unusual.  In fact, that is normal.

I am reminded of the heady early days of the space shuttle.  I was in mission control for STS-1, 2, 3, and 4; all of the ‘test’ flights.  We certainly learned much that we had not expected.

One of the areas expect differences is in the thermal response of the spacecraft.  A senior manager once remarked to me that thermal analysts, making predictions about how the temperatures would respond in space, should all wear ‘tall peaked hats decorated with moons and stars’.  Since I studied heat transfer seriously in engineering school, this struck home.  There are always many assumptions and complex geometries to model for thermal predictions.  For example, liquid propellant swirling inside titanium tanks held by boron struts attached to aluminum structure with mylar insulation swaddling critical components and the entire structure inside the famous heat tiles.  How hard could that be to analyze?  No wonder the heater duty cycles were considerably different in flight.

Working in the propulsion systems – reaction control thrusters and the like – and one of our main jobs was to project how much propellant would be used.  Attitude control is never pristine; thruster plumes impinge on structure and thrust is never precisely aligned with the principal axes of the vehicle.  Unusual results from attitude control thrusting should be expected.  Almost every maneuver used more or less propellant than predicted.  Those of us who were tasked with ensuring that the gas tank never ran dry before the end of the mission were frustrated by the seemingly simple to analyze phenomenon which never worked exactly as predicted.

On the third flight there were a series of extreme thermal attitudes (think bottom to sun for days) to test exactly how the structure would behave.  The shuttle got into ‘banana mode’ – just what it sounds like with the bottom expanding with the heat and the top contracting with the cold.  Then tried to close the payload bay doors.  Not surprisingly the doors would not meet, like the doors in an old rickety barn.  Equilibrium had to be reestablished before that critical operation of payload bay door closure was successful.  The flight controllers who were charged with monitoring the thermal state of the orbiter built a device to illustrate the temperature state.  This highly scientific device was a wooden base with a short wooden pole which they installed on the top of their console.  It was to be a quick visual indicator of thermal status.  Several of the flight controllers were from Louisiana and on this short wooden staff was a sliding figure of a crocodile (or was it an armadillo? Memory is misty).  When the thermal differences were high, they moved croc up; as things returned to equilibrium the croc went down.  They called this device the ‘thermo-gator’.  A little bit of flight controller humor that the management tolerated. 

Preflight much analysis was done on the navigation systems, particularly expected performance of the Inertial Measurement System (IMUs).  This is three highly accurate gyroscopes and associated accelerometers.  The IMUs were critical to maintain shuttle attitude as well as to generate updates to the all-important state vector used to calculate the shuttle position.  The gyros tended to drift which required updating by two built in telescopes called star trackers.  Having to do a star alignment to update the gyros required the shuttle to stop whatever activity and turn to at least two positions where bright stars were in the right position to check alignment.  Not so very different from using a sextant to navigate a ship at sea.  Remember the shuttle flew well before the GPS system had been built.  Predicting gyro drift and the time allowed between star alignments was studied very thoroughly pre-mission.  IMU performance turned out to be much better than predicted so the star align maneuvers could be scheduled much further apart.  Good news there.

Returning to the propulsion area, during the very first firing of the Orbital Maneuvering System engines a pressure measurement on the inlet of one engine showed a small uptick during the burn.  Not enough to cause problems but a funny thing nonetheless.  A young back room flight controller on the next shift caught that uptick and kept the data.  After the flight he made a presentation to the Orbiter Project manager asking that the fuel line into that engine be inspected – a nontrivial task.  After some discussion (a story for another day), the inspection was ordered, and the remains of a plastic bag were found on the filter to the inlet of the engine.  Not a good thing.  I knew that young flight controller well and wound up working for him later in my career.  I hear he’s still working actively at SpaceX; his name is Bill.  By the way, those OMS engines are stored for use as the main spacecraft engine for the upcoming Orion missions.  Hope we cleaned out those lines well. 

Many lessons were learned on the first shuttle flight; there are documents full of ‘lessons learned’.  But here is the point:  operating a new spacecraft for the first time is full of surprises.  Hopefully only small surprises that are easily accounted for.  Or perhaps even positive results like the shuttle IMUs.  But in the real environment of space, with its extreme conditions, mean that systems never ever operate exactly as predicted.  A vigilant operations team watches for all of these things and updates plans and procedures accordingly.

Best wishes to that professional team operating the James Webb Space Telescope! I’m impressed.

Reading List 2021 into 2022

And further, by these, my son, be admonished: of making many books there is no end; and much study is a weariness of the flesh.  – Ecclesiastes 12:12 KJV

I read a lot.  Technical journals (AIAA, ASME, ISSF), Aviation Week & Space Technology, Space News, National Geographic, Nat Geo History, Sky & Telescope, Backpacker, others.  Newspapers every day – the actual paper ones (yes, I’m that old).  And, of course, there is my work:  technical reports by the dozen, briefing papers, proposals, etc.  Most of that last is electronic; seems work has gone ‘virtual’ these days. 

But I still have time to read books.  I’m not an e-reader kind of guy; I want to hold the book in my hand and feel the heft of it, the smell of it.  I spend too much time on my electronic devices as it is. 

Looking over past years, 2021 has been typical:  space related books, history, good mysteries, a few religious books, current events, some great literature.  A few ‘re-reads’ of books read decades ago have crept in as well. 

Here is my list of books read in 2021 – in no particular order.  Not a bad book among them:

The Complete Poems and Plays of T. S. Eliot

The Cure at Troy – Philoctetes by Seamus Heaney

Giants in the Earth by O. E. Rolvaag

The Good Shepard by C. S. Forester

Hamnet by Maggie O’Farrell

The Tale Teller by Anne Hillerman

The Stargazer by Anne Hillerman

The Blessing Way by Tony Hillerman

The Dance hall of the Dead by Tony Hillerman

Talking Mysteries by Tony Hillerman and E. Bulow

Prophetic City by Stephen L. Kleinberg

Once A Warrior by Jake Wood

Once they Moved Like the Wind by David Roberts

Kearney’s March by Winston Groom

The Twentieth Maine by John J. Pullen

In the Garden of Beasts by Erik Larson

Mercury 13 by Martha Ackmann

Shuttle, Flight by Paul Dye

Go, Flight by Milt Heflin and Rick Houston

Through the Glass Ceiling to the Stars by Eileen Collins and Jonathan Ward

Liftoff by Eric Berger

The Poudre, a Photo History by Stanley R. Case

The Definitive Biography of P.D.Q. Bach by Prof. Peter Schickele

Lincoln on Leadership by Donald T. Phillips

In a Pit with a Lion on a Snowy Day by Mark Batterson

Looking ahead to 2022 I have already accumulated several books either purchased outright or received as gifts.  Ready to read:

Back to Earth by Nicole Stott

Not Yet Imagined – A study of Hubble Space Telescope Operations – by Christopher Gainor

50 Years of Solar System Exploration – Historical Perspectives – Linda Billings, editor

Brave Companions – Portraits in History – by David McCullough

No Barriers by Erik Weihenmayer and Buddy Levy

The Thursday Murder Club by Richard Osman

And if Anne Hillerman produces a new Manuelito mystery novel continuing the series started by her father – I will be first in line at the bookstore to get a copy.

The book I am going to read first in 2022 – taken off the shelf from my daughter’s collection, is The Count of Monte Christo by Alexander Dumas.  Everyone should have a little classic literature in their lives.  I’ve missed reading this one and need to fill that blank.  It is, so I’ve heard, a book where nothing is quite what it seems to be.  Maybe a parable for our times. 

Of all of these books, what is my most recommended? 

All of them are good.  But my pick of the year is an old one:  The Good Shepard by C. S. Forester.  It was written in 1955 and Hollywood just made it into the stunning movie “Greyhound’ in 2020. 

But the book – the book as it always does – the book reveals the internal dialog, the fears, and the courage of the principal character in a way that the silver screen with all its CGI capabilities cannot. 

Read that book for its lesson of the importance of character: “Yet these were matters of primary importance, for in war the character and personality of the leader is decisive of events much more than the minor questions of material.”

And finally, 2022 may be the time I take the advice of Eileen Collins, Homer Hickam, and many others to start writing my own book.  There may be a few stories that I need to preserve for posterity.  All mostly true, of course. 

STS-108

It has been twenty years since the flight of STS-108 also known as ISS Utilization Flight (UF)-1. 

In retrospect, 108 seems just another routine, hum drum, typical ISS resupply/crew change/ assembly mission; there were so many of them on the shuttle manifest.  Important at the time, certainly contributed to the success of the ISS, but nothing to really stand out in the record.

But the world has changed, and the comparison is, well, interesting. 

In view of the anniversary, it might be helpful to record some of my thoughts about that flight and compare them with today. 

I was the Lead Flight Director for STS-108.  My third assignment and the last to take that role.  The Lead Flight Director always started more than a year in advance of launch date to develop the flight plan.  The high-level objectives were always defined by the program office but how to accomplish them became the responsibility of the Lead Flight Director and all the flight design team.  A payload commander was also assigned early, and it was always a real team effort to put the flight together.  Dom Gorie was a great commander to work with.  Sometimes Flight Directors and the Mission Commander have friction; that was never the case here. 

STS-108 Orbit 1 Flight Control Team

My primary assignment was to be an Ascent/Entry Flight Director responsible to get the mission from the launch pad safely to orbit and then back to landing.  I served in that role for 28 flights.  But all Flight Directors had some responsibility for the on-orbit activities.  I was an Orbit phase Flight Director on even more flights. Typically, there were three or four Flight Directors to cover all the shifts on a shuttle flight:  also on 108 were Paul Hill, Cathy Koerner, Kelly Beck, and A/E FD Leroy Cain.  There were also ISS Flight Directors over in the other control room.  We all had to know how to execute a flight.  I even took some training as an ISS Flight Director.  Events conspired to take me out of the Flight Director office before I could stand for that certification. 

I have written about my other two lead flights:  my first lead and still favorite flight, STS-77   https://blogs.nasa.gov/waynehalesblog/2010/05/26/post_1274899149308/  and the first logistics flight to the ISS before it was permanently crewed, STS-96 https://blogs.nasa.gov/waynehalesblog/page/12/ 

It turns out that the most important event on STS-108 really had nothing to do with the flight but only with the timing. 

Early on the ISS Program Office asked us to plan to replace the BBRRM (pronounced ‘broom’).  The ISS gets all its electrical power from solar arrays that track the sun when it is in view and the charged batteries when its dark.  Those solar arrays rotate through the 90-minute orbit to maximize the power production by always pointing at the sun.  There is something called an Alpha joint which allows the rotation.  The SARJ is the mechanism the actually accomplishes that movement.  But over the course of longer time, months, the orbit changes.  Much like the seasons at the earth’s surface, the sun angle is lower at sometimes and higher at others; a second joint, the Beta joint, allows smaller, slower rotation to keep the Alpha joint aligned with the sun.  Too complex?  Don’t worry about it.  On the Port Side solar array number 6 the Beta joint mechanism was having problems.  Our objective was to replace the Beta Bearing Motor Roll Ring Module BBRRM – the ‘broom’.  Not an easy task because it had to be done by EVA crewmembers.  At the point the old ‘broom’ was disconnected the solar array would be no longer physically connected to the ISS.  It would be totally unrestrained until the new ‘broom’ was installed.  Another minor item, the work had to be done at ‘night’ during the half orbit when the ISS was in the Earth’s shadow.  When the sun came up the solar array would energize, and the EVA crewmembers would be subject to potential shock hazard.  The work had to be done in the dark.  In about 45-minute chunks.   

Not a little problem to solve. 

The crew, the EVA team, and I spent countless hours in meetings, wrangling over the best way to execute this task.  Then we went over the NBL – that giant water tank where the crew practiced in simulated free fall.  I inhaled lots of chlorine scented air in our months developing the EVA procedures.

Just about the time we got the procedure perfected, the ISS Program Office decided not to execute it. 

I don’t know whether I was more disappointed or relieved.  All the EVA crew members had to do now was wrap some insulation around the BBRRM which the engineering team determined that would keep it working properly.  Linda Godwin and Dan Tani got to do just one short EVA to insulate the BBRRM plus a few other minor assembly tasks. 

The mission featured an ISS crew exchange:  Yuri Onufrienko, Carl Waltz, and Dan Bursch up for a tour on the ISS; Frank Culbertson, Mikhail Turin, and Vladimir Dezhurov down as their expedition was ending.  A very international crew indeed. 

Tons of cargo – supplies for the ISS – were transferred using the MPLM (Multipurpose Logistics Module) “Raffaello”.  (We all thought the MPLMs were named for the Teenage Mutant Ninja Turtles, but officially that is a myth).  Linda Godwin and Mark Kelly – now the US Senator from Arizona – operated the shuttle RMS to move Raffaello from the shuttle cargo bay to the ISS.  And then, much later, back to the shuttle after Raffaello’s cargo was emptied and it was refilled with trash. 

External cargo like the Russian Strella ‘crane’ were also transferred and installed. 

The only excitement of the mission was the intermittent failure of one of the IMU.  Earlier in the shuttle program that would have been cause for early mission termination.  Given the shuttle experience at that point, the mission continued and was even extended a day to do some extra work at the ISS. 

All in all, it was a pretty typical, routine, hum drum, run of the mill Shuttle mission to resupply the ISS and exchange crew members. 

Except 9/11.

Honoring Heroes

STS-108 was the first flight after that terrible day.  And the three month ‘anniversary’ would occur during the flight.  Endeavour carried the flag from Ground Zero and much more.  Solemn artifacts. 

NASA leadership told the Lead Flight Director to provide an appropriate ceremony to mark the time of the ‘anniversary’. 

That suddenly became my biggest stress point of the mission. I would have much rather faced an EVA with a loose solar array. 

 I had to write some appropriate words to read up on the air/ground, arranged for the two national anthems to be played, and insisted everybody to be dressed appropriately – not that dress code was ever relaxed in the MCC anyway. 

It must have come off acceptably because I never got any feedback from the NASA leadership.  No news is good news in that regard; I expect that if it had not been appropriate, I would have promptly received significant feedback.  I have since misplaced my notes from the day, my remembrance is the remarks were pretty bland.  The flight crews of shuttle and station seemed to appreciate it.  My flight control team certainly gave me positive feedback.

In those days, the whole world was on our side.  We had sympathy and support from everywhere it seemed.  And especially from our Russian colleagues.  The partnership was tight. 

Some things have changed since then.  I wish that great rapport and support still existed. 

After the flight ended, I worked as the Mission Operations Director during the STS-109 Hubble Repair mission, spent a couple of flights assisting the A/E guys as ‘weather flight’, and then got to sit in the center seat as Ascent/Entry Flight Director for STS-113.  Did not know that my residence in the Flight Director office would end so soon after STS-108.  It was a great mission.  One for the history books. 

Like so many others.

Except for the remembrance. 

Everything I Know About Orbital Debris – Almost

Terror. 

Trembling fear and anxiety. 

My buddies decided we should each jump off the high diving board at the community pool the summer I turned eleven.  That diving board was not over 8 feet – possibly less – but at the time it seemed to be a mile high. Steeling my nerves, I climbed up the metal ladder to the platform, my way out to the end of the board, took a look down . . . and retreated back down the steps.  The sting of humiliation, not courage, made me climb up again.  This time I jumped.  Newton’s Laws took effect as my body described a ballistic path.

Free fall seemed like forever before WHACK! The surface of the pool. 

The sting of the impact is still fresh in my memory.  No permanent damage was done.  And best of all, my buddies cheered!

Public risk was more acceptable in those days. Now it is nearly impossible to find a high diving board at a community pool, even that old high board and its pool are long vanished.

 A few years later in Physics class we learned to calculate free fall and final velocity using g = 32.2 feet per second per second acceleration (9.8 meters/s/s).  Gravity pulling inexorably toward the center of the earth.  My impact occurred with a velocity of no more than 8 feet per second – barely running speed – and free fall could not have lasted more than a half second, no matter what it seemed. 

Free fall can be experienced briefly on a roller coaster, bungee jumping, parachuting, or in other high anxiety activities, but it is always brief.  Flying a really great parabola on the Vomit Comet yields almost 30 seconds of free fall before the pilot has to pull the nose up.  But Newton rules, gravity is always in control, accelerating everything at 32.2 ft/s/s directly toward the center of the earth. 

With no action to prevent the inevitable, free fall always ends with a big WHACK when the ground meets you. 

Technically it is cringe worthy to discuss the orbital experience as zero-gravity (or micro-gravity).  Newton’s infallible laws dictate that gravity has its full effect on everything in orbit.  The experience is not the absence of gravity, it is just free fall.  Why, then, is there no big WHACK at the end?  When time has passed to cover the distance down from orbit to the surface of the earth?  To be in orbit means to travel forward fast enough so that when falling the distance to the surface of the earth, the earth is not there anymore. The curvature of the globe has made the earth fallaway from you.   

The extraordinary speed necessary to do that is almost 5 miles per second (more than 7.5 kilometers per second).  Compared to our everyday experience – where walking is about 3 miles per hour; driving on a clear highway can exceed 1 mile per minute – orbital velocity is incredible, unfathomable. 

Strange and unusual things happen at such speeds.  Orbital mechanics is counter-intuitive, unlike our common experiences in almost every respect.   

Traveling at 5 miles per second, collisions between objects in orbit generate tremendous energy.  A rule of thumb states that the energy released by an orbital impact is equivalent to exploding 25 times the weight of the impactor of TNT.  A very small item can carry a huge wallop.  WHACK indeed. 

‘Low’ earth orbit – LEO – arbitrarily up to 1000 km (600 statute miles) altitude high.  Some definitions include higher altitudes but compared with the vastness of space LEO is close.  LEO is where the ISS, the Hubble Space Telescope, and many other important satellites travel.  On the low side, the lowest altitude at which an object can remain in orbit is roughly 75 miles/120 km.  Below that, atmospheric drag inevitably causes a spontaneous re-entry.  The atmosphere there is thick enough to dissipate forward speed into heat energy, soon the earth no longer curves away below you fast enough to prevent that WHACK at the end. 

Detection of orbital debris is difficult, impossible for the smallest stuff.  Many nations have sensors scattered around the world used for debris tracking.  These are primarily powerful radars but also include optical scanners and some other devices.  This loose network is not continuous but has large geographic gaps.  Many of the sensors are primarily designed and operated for other functions – such as the detection of ballistic missile launches.  Commercial and academic organizations have started developing sensor networks to detect and track orbital debris.  All these sensors have different capabilities: they vary in sensitivity, accuracy, continuity of operation.  Some of them provide only classified data which is not available the public.  For all sensors the accuracy of the measurements is degraded if the object orbits tracks nearer the horizon or at a longer distance. 

The oft-cited tracking limit is 8 cm/4.5 inches – softball size.  The exact capability is, as you might guess, classified.  The lower size limit of objects that can be tracked depends on their altitude and reflectivity. Many debris objects are smaller than the trackable limit:  metallic residue from solid rocket motor firings, paint flakes and the like.  Worse are the slightly bigger objects, still untrackable, like bolts and other metallic object shed when a rocket releases its payload in orbit.  As modern CubeSats reduce in size, difficulty to track them increases; the smallest CubeSats in LEO are trackable only if their radio transmitter is active.  Only the big stuff in LEO is easily tracked:  large satellites either active or derelict, upper stage rocket bodies expended but still in orbit.  The catalog of trackable objects in LEO numbers nearly 10,000.  It has been estimated that there are over one million debris objects greater than 2 mm in LEO.  A metal part 2 mm in size is a very dangerous object traveling at 5 miles per second. 

 All the tracking sensors combined constitute a porous net. A new object can arrive in such an orbit that takes hours to pass over an active sensor for detection.  After the initial detection, it takes additional time to get a reasonable track on a debris object.  To get a really good position and velocity – a state vector – requires multiple passes over multiple sensors.  Depending on the circumstances, getting a good state vector on a new orbital target can take hours to days to weeks. 

Of course, the point is to be able to predict when a collision – a conjunction – will occur.  The process of orbital propagation and results in an ephemeris or series of predictions where the object will be at future times.  Hopefully the prediction can be made far enough in the future to do something about it.    

Any uncertainty or error in the state vector will mean that a prediction of where the object will be in the future is inaccurate.  If the velocity has an uncertainty of 1 foot per second (.3 meter per second) – an error of one part in 25,000 – the position prediction an hour hence will be off by half a mile or almost a kilometer.  That error can mean the difference between a hit and a miss.  Close only counts in horseshoes.  The more precisely the current velocity of an object can be measured the better a future prediction of location will be.  All state vectors have some uncertainty. 

The process of orbital propagation is tricky.  The Space Shuttle orbit was a hard subject to propagate because the shuttle had so active thrusting activities; water vents, flash evaporator exhaust, uncoupled attitude thruster firings – all these acted to change the state vector – the position and velocity of the Space Shuttle over time.  Any vehicle with reaction control thrusters or a vent of any kind will be hard to propagate because the errors will grow fast and unpredictably. 

Don’t forget that there are gravity differences as an orbit passes over parts of the earth because it is not a perfectly uniform sphere.  It is fatter at the equator than over the poles and the substructure of the earth has regions of denser material and others of lighter.  Those differences factor into the ephemeris calculation. 

An orbiting object is further affected by the solar wind, fluctuations in the earth’s magnetic field, and even the pressure of the sunlight.  The largest effect is the variation in the density of the upper atmosphere.  It is sound strange to talk about density of the atmosphere where the vacuum is better than anything that can be created in a laboratory on the earth.  Those few widely separated atmospheric molecules that are struck by an orbiting object determines the rate at which the orbit decays.  Atmospheric density is hard to predict, the solar sunspot cycle alternately heats and cools the atmosphere at high altitudes making its density vary with time and location.  Atmospheric density at LEO cannot be measured directly and only roughly predicted. 

Orbital decay due to atmospheric drag depends on an object’s ballistic coefficient; a function of its surface area and density.  It stands to reason that a larger lightweight object – say a fabric thermal blanket – will be slowed down by atmospheric drag faster than a smaller denser object like a steel bolt.  Interestingly an object’s ballistic coefficient can change if the attitude is controlled – that blanket edge on to the direction of travel has a different ballistic coefficient than if it presents a full face to the ‘wind’. 

Orbital decay works on everything in LEO, it just takes on time.  At 350 miles high the Hubble Space Telescope has maybe three decades of orbital life after its last reboost.  At 600 miles high, a high ballistic coefficient object may hold out for almost a century.  Lower, at 100 miles high orbital lifetime may be only days.  If the orbit is not circular but elliptical, the most drag comes at the lowest point.  The perigee matters. 

Given all of this, it is not hard to see why predicting upcoming collisions between objects in LEO is messy and sometimes imprecise.  Given the uncertainties, it has become standard practice to analyze any predicted collision – or close pass – to mathematically determine the probability of actual contact.  Usually, the number is really small.  But not always.

The Two-Line Elements (TLE) that are made public to describe an object’s orbital state have little of that uncertainty information.  In the official world, there is much more information to go into the calculation. 

In the case of the ISS, if a conjunction is predicted and it falls into a highly likely probability of collision, the team springs into action.  Well, springs may not be truly descriptive.  Preparations proceed slowly.  The ISS uses thrusters on the Russian segment to provide translation – to move away from a predicted collision.  It doesn’t take much thrust to move the propagated position out of harm’s way given the given several hours to a few days between the DAM (debris avoidance maneuver) and the time of closest approach.  But the Russian segment is old fashioned and requires a specific uplink command to fire the engines.  Doing the math, building the command, and verifying it is correct can take several hours.  Nothing very fast in this process.  So, if a conjunction is predicted in a very short time, well, we all hope it’s a big sky and the uncertainties fall in your favor. 

In such a case, the ISS crew is directed to the ‘Safe Haven’ procedure.  Doing things like closing hatches between modules, putting the crew in their pressure suits, and having them hang out in their lifeboat return capsules is the best that can be done.  As Capt. Young used to term some of our Shuttle abort procedures ‘it’s something to do while you’re waiting to die.’  Safe Haven is the best that can be done, it might even be of some use.  But don’t kid yourself.

Many trackable objects are derelict and have no translation capability, no way to command movement out of them.  Some trackable items do have maneuver capability and can be commanded to move out of the way of a collision.  A couple of derelict inert satellites collided a few years ago and put a cloud of debris at the higher reaches of LEO where it will slowly descent for decades posing a problem for everybody. 

For the very smallest debris – paint flakes and the like – resulting damage from an impact is acceptably small.  There were pits found post flight in the shuttle windows on a regular basis.  Damage of this kind is considered acceptable.

For slightly larger items, but still untrackable, the ISS has debris shields – called Whipple shields after the astronomer that proposed them before the start of the space age.   These multi-layer structures protect all the critical pressurized volume of the ISS, but not its appendages like solar arrays and thermal radiators.  None of that on an EVA suit, there is very little protection for space walkers working outside the station. 

There is a larger size category of still untrackable debris that can defeat the Whipple shields and cause a hole – maybe a big hole – in the ISS.  Or any other satellite in LEO.  And by analysis and some indirect measurements there is a lot of that stuff in orbit.  So, what is the defense? 

It’s a big sky.

A debris collision is part of the ‘accepted risk’ of spaceflight in LEO.  Hope you don’t get hit.  So far, we’ve been pretty lucky.  Will that be the case in the future?  Quien sabe?  Somebody has calculated the probability of that.  But I wouldn’t use that number to place a bet in Las Vegas.

Which brings us to the popular idea to clean up the mess up there.  Certainly, it would be technically possible, with significant expenditure, to launch a fleet of garbage gatherers and deorbit a number of the derelict big stages or defunct satellites that we can track.  But the price tag will be bigger than you might think.  There are a large number of those items in LEO, and they are really spread out.  It is unlikely that a debris removal spacecraft could actually rendezvous and get rid of more than two or three targets.  But their removal may be key to avoiding the Kessler Syndrome.

Trickier than orbital mechanics is space law.  Seems that dead satellites still belong to the nation that launched them.  Unlike maritime law, salvage of derelict craft in orbit is not allowed.  Grabbing somebody else’s satellite even to clear it out as trash is illegal.  In the worst case, it could be considered an act of war.  Think about that. 

As for the small stuff that can’t be tracked but could still cause catastrophic damage – it is simply too widely spread out to easily clean up.  Space is big, even at LEO.  The very simplistic idea of a fleet of spacecraft going around the earth sweeping up all of that stuff is, well, impractical.  But somebody may try, bless them for the attempt.  I hope I’m wrong, but it is a lot simpler and cheaper to clean up the Great Pacific Garbage Patch than Low Earth Orbit.  Nobody is seriously attempting either. 

The best way now to control space debris is to insist that upper stages, when they have finished their job, retain enough propellant to immediately deorbit themselves.  And likewise, provision for satellites at the end of their life should be made to lower the orbit so that they naturally decay in relatively short times.  At the very least, propellant tanks should be vented, and batteries discharged because sometimes those items deteriorate over time and cause the object to explode spontaneously. 

On the hopefully rare occasions when a rocket stage or satellite explodes because its batteries degraded or its propellant tanks leaked, or if there is a collision, or if – hopefully never again – somebody tries to shoot down a satellite; the pieces will be flung all about the original orbit.  They may go somewhat higher or lower, left or right, farther ahead or slightly behind.  This cloud of debris will slowly expand over time as the residual velocity differences of each piece spreads them apart.  None of the collision products will ‘fall to earth’ right away.  It gets to be a complex problem.

And one of the controls should be, of course, making sure there are no future ASAT (anti-satellite missile) tests in low earth orbit.  Good luck with that. 

But if you ever heard the talk, like I did, by a fellow named Don Kessler, you would worry about the consequences of increasing orbital debris.  Don held out the hope, still true, that if at least some of the bigger derelict objects could be eliminated, the space lanes could be kept open.  Because if collisions start coming more frequently and the debris multiplies; well, they named it after him:  The Kessler Syndrome.  It would be bad.  The loss of access to space is in the balance.   

Last thing we need in orbit is another WHACK.

I have a lot of fear and trembling over that. 

https://blogs.nasa.gov/waynehalesblog/2009/12/09/post_1260386290939/

sAnd after nearly forty years in the space business, that is really about all I know about orbital debris.  Mostly. 

Main Engine Controllers

Recent unconfirmed media rumors about engine controller issues brought a long-ago memory to the surface.

About 1991, the Chief of the Flight Director office decided that it would be good to have an exchange program between the Flight Directors and both the NTDs and Convoy Commanders.  At that time, I was a new Shuttle Ascent/Entry Flight Director with just a couple of critical phase flights under my belt.  In turn, each of the Shuttle Convoy Commanders came to JSC to sit with an Entry Flight Director for a landing, and then each Entry FD took a turn sitting with in the Convoy Commander vehicle during a Shuttle landing.  That was where I got to know Tassos Adiabakos, Kelvin Manning, and others.  My turn was for a Shuttle landing at Edwards.  Waiting for the deorbit call with iffy weather, I boldly predicted that Lee Briscoe would delay landing for a day rather than divert to the secondary target of KSC.  When Lee did just the opposite, well, it was a long time before I heard the end of it. 

Similarly, The KSC NTDs (NASA Test Directors) came sequentially to sit with the Ascent Flight Director for a launch, and we returned the favor by going to KSC and observing the action in the Firing Room for a launch countdown.  In July, I followed Al Sofge for STS-43.  We went through the pre-launch planning; Al was a perfectionist making sure the checklists were all in order, the firing room manning was prepared, and even details down to the scheduling of the janitorial crew cleaning the bathrooms – don’t want a delay because the team couldn’t get in and out of those facilities in a hurry! 

I got the opportunity to ride in the Shuttle Training Aircraft as mission commander John Blaha shot approaches to the Shuttle Landing Facility.  It is quite a ride on the jump seat in the cockpit watching the runway rush up at you while the engines scream in full reverse.  With the flaps and dive brakes out, the aircraft still accelerates in a steep approach.  What a rush!

Finally, on launch morning, there was no spot for me to sit on the NTD row of the firing room, so they put me up on the top row, with the legendary Launch Director Bob Sieck.  I felt like a padawan under the tutelage of Obi-Wan.  Bob was as smart as they came, cooler than ice water, never fazed about anything.  He had been a meteorologist in the Air Force before coming to NASA so the toughest call the Launch Director has to make – is the weather good enough – was not problem for him. 

It was to be an early morning launch, so the pre-launch action all took place in the wee hours of the night.  We got a go for tanking and the cryogenics started flowing into the Shuttle, chilling down the propellant systems:  tanks, plumbing, and the engines.  About the same time the smell of cornbread and beans warming upstairs started wafting through the Firing Room ventilation system. Talk about launch pressure!  https://waynehale.wordpress.com/2019/03/03/launch-fever/

Suddenly, the electronic brains of one of the three Space Shuttle Main engines winked off.  Just like that, with no notice.  It was dead, no activity, no signals, no nothing.  It was as if all the redundant power feeds had been switched off at once. 

This was not good.

Clearly this was a violation of the Launch Commit Criteria.  With the SSME Controller failed, the engine could not start.  At T-31 seconds when the onboard Redundant Set General Purpose Computers took control, they would immediately halt the launch sequence. 

The phone rang – it was the Space Shuttle Program Manager Brewster Shaw. 

I learned a lot from Brewster – he had high standards and always ‘encouraged’ the team to lean forward but was always considerate of crew risk.  Al Shepard was called ‘the icy commander’ but he had nothing on Brewster.  I found him was very intimidating.  He wanted to launch anyway.  I was appalled at the time.  Now, I’m not so sure. 

It’s a complex thing to launch a spacecraft, doubly so for the Shuttle.  As Brewster’s successor at the Shuttle Program Manager job, Tom Holloway, frequently told us, ‘the hardest part of a shuttle flight is getting the first foot off the ground.’ 

Later, when it was my turn to hold that position, I frequently about Brewster and Tommy and what it took to get the team to commit to get that first foot off the ground.  But that night I was a rookie and shocked at what Brewster asked.  He wanted to know if there was any way to reset the engine controller and try to launch.  Wise old Bob Sieck shook his head and said probably not, but the team will think about it.  Brewster recommended cycling the power switches to see if the computer would come back on.  But we all knew that even if the computer restarted, it probably couldn’t be trusted to perform properly all the way to orbit. Likely nobody would be comfortable with that possibility in just a few hours.  But Brewster said try, so the team did.

No joy.  SSME controller was still dead, kaput, nonfunctional.  No question about what to do.  Shortly afterward the launch scrub was announced.  John Blaha and his crew hadn’t even had breakfast yet. 

The team at KSC figured out a way to change out the controller on the engine while the shuttle was on the launch pad.  Just over a week later Atlantis launched flawlessly.  But I wasn’t in the firing room.  My travel allowance timed out.  I watched from the assistant flight director seat in the MCC.  I actually liked that better:  better displays there for the flight but the price was less fire and smoke out the window.  And I never got to eat the beans post launch STS-43. 

It is a terrible pressure to be a Program Manager trying to nudge the team into flying all the while making sure that safety is not eroded.  It didn’t hurt anything to power cycle that computer.  I sometimes wonder what we would have done if it had come back on.

Brewster changed the Shuttle paradigm which was often stated of Fly Safely to a slightly different phrase: Safely Fly.  Think about the subtly of the message when put that way.  Brewster put safety first.  But he always challenged the team when he thought there was a way to get off the ground. 

So did I when it was my turn. 

By the way, when they got that Engine Controller into the shop and opened it up, a power cable had broken.  No doubt due to contraction of the device as the engine chilled down.  No way that computer was going to run. 

The new RS-25 Engine Controllers are more reliable and resilient

But it is still hard to get a rocket the first foot off the ground.

Exploding Stovepipes

stove·pipe  /ˈstōvpīp/  noun

noun: stovepipe; plural noun: stovepipes; noun: stove-pipe; plural noun: stove-pipes

1. the pipe taking the smoke and gases from a stove up through a roof or to a chimney.
2. an information conduit that traverses vertical levels efficiently but does not disperse widely.

Are you ready to go down a detailed technical rabbit hole?  Hang on because there are lessons to be learned from this at the end.  I suppose that I should have tried to summarize this story in a more abbreviated manner, but I have come to the conclusion that missing the richness of the details in a history lesson limits the lessons that can be learned.  So, it is long.  Back to the story: 

The Space Shuttle was held down on the launch pad with eight huge nuts – 35 or so pounds about 8 inch diameter – at the bottom of the two solid rocket boosters.  These nuts were screwed onto threaded steel studs about 3 inches in diameter, roughly 6 feet long. Those studs, connected to the launch pad and placed under enormous tension held the shuttle stack during roll out to the pad and during that famous ‘twang’. Many people, looking at the launch pad, erroneously concluded that the three-story tall tail service masts on either side of the orbiter also provided support.  The TSMs did not, they merely connected cables, flexible hoses for various fluids and gases.  No support was provided.  All the weight of the combined stack – and the famous ‘twang’ or lean of the stack that occurred as the main engines ran up before the boosters fired – all those loads were transmitted through those eight bolts and the threaded steel studs from the launch pad.  None of these items were small.  . 

The shuttle onboard computers issued the commands for multiple pyrotechnic devices all in the same minor cycle of software (80 milliseconds):  SRB ignition, hold down bolt separation, GUCP separation, TSM/T0 umbilical separation.  Things started happening fast after that. 

The massive hold down bolts had two sets of explosives 180 degrees apart which separated the nut into two halves in a symmetric trajectory designed to propel the halves away from the hold down stud.  Tension on the hold down stud should immediately cause it to retract into the housing in the launch pad while the bolt fragments were contained in ‘bolt catcher’ devices to made the ride up nearly to space and down to the ocean with the SRBs.  A failure of the nut to separate was considered catastrophic because that event would severely damage the aft skirt of the SRB where the hydraulics and steering mechanism for the nozzle were contained.  Thankfully, such a failure never occurred during the shuttle program.  But what was observed, on 25 or so launches, was a problem where the nut did not separate cleanly and the threaded stud was pulled up out of its housing for a few seconds. 

What is maybe even more obscure is the consequence of this so called ‘stud hang up’.  A shock wave – imperceptible to view – travels through the system when that stud lets loose.  No deviation to the trajectory.  No damage to the aft skirt of the SRB other than some cosmetic scratches.  No damage to the attachment hardware that connected the SRB to the ET carrying the huge liftoff loads as those SRBs lift the fully fueled stack.  No damage to the ET.  No damage to the attachment hardware that connected the ET to the Orbiter. But deep inside the orbiter, analysis indicated the shock wave from the hang up release could cause significant structural damage.  In some cases, analysis indicated that the structure holding the vertical tail on the orbiter could be over-stressed.  Leaving the shuttle orbiter tail on the launch pad would be, well, catastrophic. 

The critical level of shock could not happen if just one stud hung up, nor if two studs hung up, it might happen if three studs hung up and released in a certain sequence with certain limited wind conditions, but the problem was certain to be critical if four or more studs hung up. 

The shuttle program decided to ‘monitor’ the stud hang ups.  Observe whether they occurred, and if they did how frequently and how many on a given flight.  Hang ups occurred infrequently, rather like the O-ring erosion in the early solid rocket segments or major foam losses from the ET.  Monitoring was considered a viable technical way to control the problem.  If one or two stud hang ups occurred – two hang ups occurred on two flights – it was worrisome but OK.  If three ever occurred, the shuttle program promised to fix the problem.  And so, the troublesome design set on the back burner, so to speak, simmering but not rising to the top of the priority list to fix.  In retrospect, this is nothing more than playing Russian roulette.  It was not a control.  It was whistling in the dark.  Not an acceptable management or technical protocol.  Lesson 1:  Do not do this. 

Why does stud hang up occur sometimes and not all the time?  There are two pyrotechnic devices on the bolt which are subject to very slight delays in the firing circuit.  The pyrotechnic operation can occur a few milliseconds apart, with the result that one side of the nut opens slightly before the other side opens.  In some cases, this forces the stud up against the side of the aft skirt.  In some cases, one of the nut fragments holds to the stud threads for a few seconds.  Frequently, normal separation with no hang up occurs.  But the bottom line is that we knew we had a problem, we knew it could be serious, and we knew what caused it.  Turns out, we also knew how to fix it.   

After the loss of Columbia, there was a review of all the lingering problems with the shuttle system.  We put together plans to fix as many as we could.  This stud hang up was a problem that should be fixed, we decided. Marshall Spaceflight Center and the solid rocket booster project were tasked to improve their pyrotechnic devices to eliminate the potential for a stud hang up.  The solution was surprisingly simple: merely cross strap the charges right there on top of the nut.  This eliminated the delay, the off-kilter separation, and thus eliminated the cause of the problem.

For twenty years, the shuttle program folks had studied this problem and spent untold engineering hours trying to analyze permutations and combinations of hang ups and what effect this might had. Every Shuttle launch had a probabilistic risk analysis for this failure.  The program spent millions of dollars in engineering manhours studying this problem each flight.

During the redesign, I went to the Marshall Spaceflight Center where they were testing this cross-strap bolt.  Testing is important because any new design must be demonstrated to work and not cause unforeseen problems before it could be certified for flight. Watching some of the testing, I talked to the people on the floor who surprised me by reporting: ‘You know what Mr. Hale?  We had this cross-strap bolt ready to go in 1984. It was almost certified. We almost put it into work then.’  I was flabbergasted. For over twenty years the shuttle program had been living with this problem. Why was this improved design not implemented then? In response the workers noted that stud hang up does not cause any problems to the solid rocket booster other than cosmetic damage inside the skirt.  And the budget was cut on the SRB project – as all the shuttle projects budgets were cut in the middle 80’s. The shuttle was operational after all; cost was a problem.  Design and development were over, surely those costly engineers could be taken off the program and no new redesigns were required, right?  Something had to go and in the calculus of the SRB project manager stud hang up was not causing the solid rocket booster project any problem.  So, work to complete the cross-strap pyrotechnics testing and certification was eliminated as low priority work.  Folded in with other savings the cost reduction was summarized and reported to the Program Manager and he was pleased.

And so, for twenty years, another part of the program paid untold millions of dollars in engineering analysis.  And the whole system was at risk every single launch.   

This is, after all, rocket science. 

So, grasshopper, what can we learn?  I have a few lessons and you may be able to glean more.

1.  Arbitrary budget cuts in high technology closely coupled systems can have unforeseen consequences.  A good program manager will delve deep enough into each work item deleted to understand the consequences of its deletion.  This takes time.  Take the time.
2. Organizations that work on complex technologies are often, by necessity, broken into work elements to get the job done.  Overview management must be strong enough to detect when one part of the organization makes a decision that will have consequences in another part of the organization.  Stated another way, in space flight most failures occur at the interfaces – technical and organizational. Be wary of stovepipes.
3. Space flight vehicles are by definition experimental.  They do not fly sufficient times to build up a strong data base like other transportation systems do.  Declaring that a system is ‘operational’ does not mean that engineering vigilance can be significantly reduced.  Bean counters always assume the engineering support withers away after the development phase is over.  This is not true for space flight systems; they carry large engineering workforces into the operational life of the vehicle for good reason.  Beware of program budget plans that assume large cost reductions when the vehicle is declared ‘operational.’  These devices may have an operational mission but from the engineering standpoint, they will always be ‘experimental.’
4. Monitoring a problem is never the right answer. Fixing the problem is.

And one more for the policy makers:

Whenever we try to do something high risk and complicated – and spaceflight is terribly complicated – especially when the technical systems are highly coupled with low factors of safety, we will face problems like this.  John F. Kennedy spoke out about this when he proposed sending Americans to the Moon.  He added that if the nation were to decide to go to the Moon and stop because it becomes too expensive or too dangerous, it would be better not to go at all.

These systems are expensive, I am sorry. I wish it were different. Many people are working to reduce costs, but my experience is that they will remain expensive, at least with our current technology.  Space projects must be fully supported by adequate resources. Or it would be better not to start at all. 

The Changing of the Guard

These days I take a lot of delight in my grandchildren.  I have some pictures here . . . .  Well, maybe not now.  I also greatly enjoy my children – grown to adulthood; we have great conversations and they actually seek out my advice from time to time.  Best of all is when I hear them say words to their children that I used to say to them.  Yes, they remembered even though at the time I wondered.

Likewise, while I am not totally retired, I am heading that way and my workload is much less than it used to be.  So, it gives me great pleasure to see the changing of the guard at my old workplace.  Many of the folks in new positions worked with me, for me, or were mentored by me in their early careers.  I listen to their press conferences or read their written remarks and sometimes I can even hear echoes of phrases that they probably heard from me long ago.

It is great to see the energy and new perspectives at work on some of the same old problems that need to be solved.  I can remember the ambition and the burning eagerness to get things done and move the organization along.  Still lurks down there somewhere but not burning as bright as it does in the younger folks.

As an older white guy, it also gives me a really positive hope for the future in the diversity now exhibited in leadership roles.  Something I learned early is that diversity and the exchange of ideas frequently leads to better outcomes than when the team is all alike.  Many more positions are filled by women (inherently smarter than men, I think) but more needs to be done to gain the perspective that people of color bring.  It’s a societal problem that reflects itself in a technical work force.  More mentoring and opportunities down to the grade school level need to be available.

So, I’m happy that we will get to the stars together; the energy and innovation is coming together.  Meanwhile, they even sometimes actually ask for my advice.  And that give me a lot of pleasure, too.

Best of luck to them!  I feel confident in the future because of what I see.