Exploding Stovepipes

stove·pipe  /ˈstōvpīp/  noun

noun: stovepipe; plural noun: stovepipes; noun: stove-pipe; plural noun: stove-pipes

1. the pipe taking the smoke and gases from a stove up through a roof or to a chimney.
2. an information conduit that traverses vertical levels efficiently but does not disperse widely.

Are you ready to go down a detailed technical rabbit hole?  Hang on because there are lessons to be learned from this at the end.  I suppose that I should have tried to summarize this story in a more abbreviated manner, but I have come to the conclusion that missing the richness of the details in a history lesson limits the lessons that can be learned.  So, it is long.  Back to the story: 

The Space Shuttle was held down on the launch pad with eight huge nuts – 35 or so pounds about 8 inch diameter – at the bottom of the two solid rocket boosters.  These nuts were screwed onto threaded steel studs about 3 inches in diameter, roughly 6 feet long. Those studs, connected to the launch pad and placed under enormous tension held the shuttle stack during roll out to the pad and during that famous ‘twang’. Many people, looking at the launch pad, erroneously concluded that the three-story tall tail service masts on either side of the orbiter also provided support.  The TSMs did not, they merely connected cables, flexible hoses for various fluids and gases.  No support was provided.  All the weight of the combined stack – and the famous ‘twang’ or lean of the stack that occurred as the main engines ran up before the boosters fired – all those loads were transmitted through those eight bolts and the threaded steel studs from the launch pad.  None of these items were small.  . 

The shuttle onboard computers issued the commands for multiple pyrotechnic devices all in the same minor cycle of software (80 milliseconds):  SRB ignition, hold down bolt separation, GUCP separation, TSM/T0 umbilical separation.  Things started happening fast after that. 

The massive hold down bolts had two sets of explosives 180 degrees apart which separated the nut into two halves in a symmetric trajectory designed to propel the halves away from the hold down stud.  Tension on the hold down stud should immediately cause it to retract into the housing in the launch pad while the bolt fragments were contained in ‘bolt catcher’ devices to made the ride up nearly to space and down to the ocean with the SRBs.  A failure of the nut to separate was considered catastrophic because that event would severely damage the aft skirt of the SRB where the hydraulics and steering mechanism for the nozzle were contained.  Thankfully, such a failure never occurred during the shuttle program.  But what was observed, on 25 or so launches, was a problem where the nut did not separate cleanly and the threaded stud was pulled up out of its housing for a few seconds. 

What is maybe even more obscure is the consequence of this so called ‘stud hang up’.  A shock wave – imperceptible to view – travels through the system when that stud lets loose.  No deviation to the trajectory.  No damage to the aft skirt of the SRB other than some cosmetic scratches.  No damage to the attachment hardware that connected the SRB to the ET carrying the huge liftoff loads as those SRBs lift the fully fueled stack.  No damage to the ET.  No damage to the attachment hardware that connected the ET to the Orbiter. But deep inside the orbiter, analysis indicated the shock wave from the hang up release could cause significant structural damage.  In some cases, analysis indicated that the structure holding the vertical tail on the orbiter could be over-stressed.  Leaving the shuttle orbiter tail on the launch pad would be, well, catastrophic. 

The critical level of shock could not happen if just one stud hung up, nor if two studs hung up, it might happen if three studs hung up and released in a certain sequence with certain limited wind conditions, but the problem was certain to be critical if four or more studs hung up. 

The shuttle program decided to ‘monitor’ the stud hang ups.  Observe whether they occurred, and if they did how frequently and how many on a given flight.  Hang ups occurred infrequently, rather like the O-ring erosion in the early solid rocket segments or major foam losses from the ET.  Monitoring was considered a viable technical way to control the problem.  If one or two stud hang ups occurred – two hang ups occurred on two flights – it was worrisome but OK.  If three ever occurred, the shuttle program promised to fix the problem.  And so, the troublesome design set on the back burner, so to speak, simmering but not rising to the top of the priority list to fix.  In retrospect, this is nothing more than playing Russian roulette.  It was not a control.  It was whistling in the dark.  Not an acceptable management or technical protocol.  Lesson 1:  Do not do this. 

Why does stud hang up occur sometimes and not all the time?  There are two pyrotechnic devices on the bolt which are subject to very slight delays in the firing circuit.  The pyrotechnic operation can occur a few milliseconds apart, with the result that one side of the nut opens slightly before the other side opens.  In some cases, this forces the stud up against the side of the aft skirt.  In some cases, one of the nut fragments holds to the stud threads for a few seconds.  Frequently, normal separation with no hang up occurs.  But the bottom line is that we knew we had a problem, we knew it could be serious, and we knew what caused it.  Turns out, we also knew how to fix it.   

After the loss of Columbia, there was a review of all the lingering problems with the shuttle system.  We put together plans to fix as many as we could.  This stud hang up was a problem that should be fixed, we decided. Marshall Spaceflight Center and the solid rocket booster project were tasked to improve their pyrotechnic devices to eliminate the potential for a stud hang up.  The solution was surprisingly simple: merely cross strap the charges right there on top of the nut.  This eliminated the delay, the off-kilter separation, and thus eliminated the cause of the problem.

For twenty years, the shuttle program folks had studied this problem and spent untold engineering hours trying to analyze permutations and combinations of hang ups and what effect this might had. Every Shuttle launch had a probabilistic risk analysis for this failure.  The program spent millions of dollars in engineering manhours studying this problem each flight.

During the redesign, I went to the Marshall Spaceflight Center where they were testing this cross-strap bolt.  Testing is important because any new design must be demonstrated to work and not cause unforeseen problems before it could be certified for flight. Watching some of the testing, I talked to the people on the floor who surprised me by reporting: ‘You know what Mr. Hale?  We had this cross-strap bolt ready to go in 1984. It was almost certified. We almost put it into work then.’  I was flabbergasted. For over twenty years the shuttle program had been living with this problem. Why was this improved design not implemented then? In response the workers noted that stud hang up does not cause any problems to the solid rocket booster other than cosmetic damage inside the skirt.  And the budget was cut on the SRB project – as all the shuttle projects budgets were cut in the middle 80’s. The shuttle was operational after all; cost was a problem.  Design and development were over, surely those costly engineers could be taken off the program and no new redesigns were required, right?  Something had to go and in the calculus of the SRB project manager stud hang up was not causing the solid rocket booster project any problem.  So, work to complete the cross-strap pyrotechnics testing and certification was eliminated as low priority work.  Folded in with other savings the cost reduction was summarized and reported to the Program Manager and he was pleased.

And so, for twenty years, another part of the program paid untold millions of dollars in engineering analysis.  And the whole system was at risk every single launch.   

This is, after all, rocket science. 

So, grasshopper, what can we learn?  I have a few lessons and you may be able to glean more.

1.  Arbitrary budget cuts in high technology closely coupled systems can have unforeseen consequences.  A good program manager will delve deep enough into each work item deleted to understand the consequences of its deletion.  This takes time.  Take the time.
2. Organizations that work on complex technologies are often, by necessity, broken into work elements to get the job done.  Overview management must be strong enough to detect when one part of the organization makes a decision that will have consequences in another part of the organization.  Stated another way, in space flight most failures occur at the interfaces – technical and organizational. Be wary of stovepipes.
3. Space flight vehicles are by definition experimental.  They do not fly sufficient times to build up a strong data base like other transportation systems do.  Declaring that a system is ‘operational’ does not mean that engineering vigilance can be significantly reduced.  Bean counters always assume the engineering support withers away after the development phase is over.  This is not true for space flight systems; they carry large engineering workforces into the operational life of the vehicle for good reason.  Beware of program budget plans that assume large cost reductions when the vehicle is declared ‘operational.’  These devices may have an operational mission but from the engineering standpoint, they will always be ‘experimental.’
4. Monitoring a problem is never the right answer. Fixing the problem is.

And one more for the policy makers:

Whenever we try to do something high risk and complicated – and spaceflight is terribly complicated – especially when the technical systems are highly coupled with low factors of safety, we will face problems like this.  John F. Kennedy spoke out about this when he proposed sending Americans to the Moon.  He added that if the nation were to decide to go to the Moon and stop because it becomes too expensive or too dangerous, it would be better not to go at all.

These systems are expensive, I am sorry. I wish it were different. Many people are working to reduce costs, but my experience is that they will remain expensive, at least with our current technology.  Space projects must be fully supported by adequate resources. Or it would be better not to start at all. 

The Changing of the Guard

These days I take a lot of delight in my grandchildren.  I have some pictures here . . . .  Well, maybe not now.  I also greatly enjoy my children – grown to adulthood; we have great conversations and they actually seek out my advice from time to time.  Best of all is when I hear them say words to their children that I used to say to them.  Yes, they remembered even though at the time I wondered.

Likewise, while I am not totally retired, I am heading that way and my workload is much less than it used to be.  So, it gives me great pleasure to see the changing of the guard at my old workplace.  Many of the folks in new positions worked with me, for me, or were mentored by me in their early careers.  I listen to their press conferences or read their written remarks and sometimes I can even hear echoes of phrases that they probably heard from me long ago.

It is great to see the energy and new perspectives at work on some of the same old problems that need to be solved.  I can remember the ambition and the burning eagerness to get things done and move the organization along.  Still lurks down there somewhere but not burning as bright as it does in the younger folks.

As an older white guy, it also gives me a really positive hope for the future in the diversity now exhibited in leadership roles.  Something I learned early is that diversity and the exchange of ideas frequently leads to better outcomes than when the team is all alike.  Many more positions are filled by women (inherently smarter than men, I think) but more needs to be done to gain the perspective that people of color bring.  It’s a societal problem that reflects itself in a technical work force.  More mentoring and opportunities down to the grade school level need to be available.

So, I’m happy that we will get to the stars together; the energy and innovation is coming together.  Meanwhile, they even sometimes actually ask for my advice.  And that give me a lot of pleasure, too.

Best of luck to them!  I feel confident in the future because of what I see.

Will the World Go Back to Normal?

You cannot step twice into the same river – Heraclitus 540-480 B.C.

Heraclitus says that we can’t ever step in the river twice; between footsteps the river changes – banks are washed away, new sandbars are formed, now more and later less water flows in from tributaries.  But probably more profound is that we are not the same.  Our bodies change, our minds change, we are transformed. Experience changes us; time changes us; we know more today than we did yesterday, or at least we hope so.

Anyway, you must define ‘normal’.

Epidemics of all sorts are ‘normal’.

I missed most of the polio scare of the 1940’s and 50’s, but I can remember my mother telling me one hot summer day that we were not going to go to the community swimming pool because there were ‘germs’.  Turns out that my parents and their generation were terrified of polio – and with good reason.  It was a long wait for a vaccine.

Going back, I had an uncle who died as an infant in 1935 because there were no antibiotics when he got sick.  Just a few years later, with penicillin and sulfa drugs he might have survived.

Further back, four of my great uncles/aunts died in a measles epidemic in 1907.  My grandfather just barely escaped that epidemic; fortunate for me that he lived and carried on the family.   My great-grandmother died in 1917 during the Spanish Flu epidemic, but we don’t know if that was the cause.  The family tree is full of large families, often 8, 10, 12 kids; seldom did all of them survive to adulthood.

Vivid in my memory when I was about 7, going to the High School cafeteria where everybody – kids and grownups alike – were given a sugar cube in a paper cup to take.  It was the Sabin vaccine against Poliomyelitis.

Not long afterwards, on a visit to the family doctor, the nurse scraped a spot on my shoulder – there is still a small round scar there:  smallpox vaccine.

As a child I had measles, mumps, chicken pox – now they tell me to get the shingles vaccine.  Oh my.

In 1976, all of us at college lined up in the basketball gym to get the swine flu vaccine.  It was supposed to be very deadly to young adults.

To see my grandchildren when they were little, I had to get more vaccinations to ensure I didn’t carry anything into their nursery. To travel overseas I had to get more vaccinations, written down somewhere.  That and don’t drink the water.

On a family vacation my daughter hooked me with her fishing line, and I had to get a tetanus booster. That was about 15 years ago, and they tell me its time to get another one, just because.

I’m not to think that my immunity from smallpox is still active after all these decades.

Modern science and modern medicine.  Wonder if I would have lived this long without it?

A friend of mine recently quoted me to me: ‘One day you realize there is more runway behind you than ahead and your perspective on everything changes.’

For the record, I hate when people to do that.  Quote me to me.

But there is that perspective.   My mind is filled with ghosts; people and places that no longer exist.  Last fall I returned to the town where I grew up.  The elementary school I attended has been razed and a beautiful new state of the art facility has been built in its place.  Very necessary, the school was not new when I attended.  But jarring to my psyche.

I passed by the place where we used to get burgers and fries, it’s not there anymore.   I drove by the houses of my teenage friends; they are no longer living there.

I’ve watched the facilities that were so much a part of my professional career get wiped away since the end of the shuttle program.  So many more places only exist in my memory.

Even worse, so many people are gone from us even though I can see and hear them in my memory like they are still here. On that trip home I visited my mother’s grave and it was interesting to walk down the row of monuments and find all her friends there along side her.  Small town.  Brings back memories of Mom hosting bridge parties, church women’s society meetings, book and civic clubs.  All her friends are with her now and forever.

In the last few days, I received word that a high school classmate of mine passed away.  He was not the first; classmates have succumbed to so many scourges:  accidents, AIDS, drugs, cancer.  Jarring to look at the class picture and count those who are gone.  Reminders that days are short and if there is work to be accomplished it should not be postponed.

With every day, the world has changed.  Some days for the better, but always changing.

So, to the question: will things go back to the way they were?

Of course not.

The better question is, what kind of world will we make?  This, too, will pass.

What will we do with our time?  Because we too will pass.

It’s just a question of perspective.

Tracking Down Ghosts

I was recently contacted by some folks who were interested in – of all things – comments I made in a shuttle press conference some 8 or 10 years ago.  These folks were concerned that I and the shuttle team had not treated a specific risk to the space shuttle with enough rigor.  Fortunately, I was very familiar with the subject and want to fill in the record on that topic.

The space shuttle was assembled and maintained in the most stringently clean environments practical.  Whenever someone entered a shuttle cockpit the full suite of protective clothing was required.  I remember that Presidential candidate John Kerry was photographed in the full bunny suit ensemble when he visited KSC and the snarky public comments about how silly he looked helped sink his electoral ambitions.  Partially because of that, I would dodge the photographers on those few occasions that I entered a shuttle.

Despite the cleanliness precautions, every time a shuttle got to orbit there was a cloud of debris liberated into the cockpit: dust, screws, washers, bits of cleaning cloth, other fabric, miscellaneous small junk.  It was a health hazard to the crew and the filters in the air recirculation system rapidly cleaned out most of it.  The crew would install a special device shortly after arriving on orbit, it had an appropriately awful acronym:  OCAC.  Say that quickly a couple of times.  But it worked to help collect all the trash floating in the air.  (Orbiter Cabin Air Cleaner).

Whenever a new uncrewed visiting vehicle or new module is plugged into the ISS, the crew must don masks and eye protection when they first open the hatch and go inside.  Every new space seems to have its share of floating junk which had to be filtered out, which can take some time.

On one shuttle mission, a crew member lost a religious medal on a short jewelry chain.  Despite searching diligently for it both in flight and then during refurbishment on the ground the medal was never found.  A couple of flights later, the medal and its chain floated out into the mid deck after having been hidden for months and more than one trip to space and back.  Where it had been lodged, undetected for all that time, is a mystery.

In a similar way, starting with STS-1, when the shuttle payload bay doors opened in microgravity every flight, junk was seen floating out:  fabric, washers, screws, unidentified parts.  This was despite the rather extreme measures taken during pre-flight preparation (and later refurbishment) to keep the bay spotlessly clean.  During turnaround operations, folks working in and around the payload bay had to wear those goofy bunny suits and were instructed to always be on the lookout for “FOD” (Foreign Object Debris).

 

Sometimes we would see junk slowly departing from the shuttle vicinity with our cameras or the crew would report seeing something.  Every single sighting was investigated to the best of our ability.

This floating space junk was of concern for a couple of reasons; first, as I have explained before (see https://waynehale.wordpress.com/2019/09/25/oops/) the mechanisms to open, close, and latch the payload bay doors were complex and it was possible something could get in a critical spot and jam one of those critical mechanisms.  Second, and related, there was always a concern that some of that junk floating out might be actual parts of the critical mechanisms that weren’t fasted down properly and now, after loosing some screws or bolts, that mechanism might not work when needed to close and latch up everything for re-entry.

Finally, there was the concern about later running into stuff that had become part of the orbital debris cloud in low earth orbit.  Not only what the crew saw, or the onboard cameras detected, there was ground tracked debris evidence.  Almost every flight the space tracking people would call us up and say that they were tracking something – generally small – in an orbit nearby the shuttle.  This was always concerning and especially so after the loss of Columbia.  Without a lot of fanfare, we would use all our tools – mostly video cameras onboard the shuttle or on the robot arm – to look all around and see if something critical – like a thermal protection tile – had come off.  I can’t tell you how many times we did such an inspection, but my guess is probably 1 of every 3 flights had an incident that resulted in as thorough an inspection as we could make.

Generally, if the tracked items were co-orbiting with the shuttle, the relative velocity was small, and not really a concern for collision damage.  But every sighting was a major concern as a possible loss of something critical from onboard.

Of course, for subsequent shuttle flights, coming in a different orbit, collision damage was a real issue. Items lost off the shuttle became part of the orbital debris environment.  If the ground tracking folks predicted a close approach we would maneuver, or not (see https://blogs.nasa.gov/waynehalesblog/2009/12/09/post_1260386290939/).

Despite all our efforts to identify most of these space junk sightings, few of them were conclusively identified.   Lots of effort was expended and a real level of concern remained over those that were not positively identified.  Unfortunately, in those days, tracking of small items in LEO was problematic.

As I started out this post by saying that recently I have been contacted about pictures of junk near the shuttle – taken long ago now.  Implied in their contact was that I was cavalier about these issues and did not pay enough attention to them, or that some shadowy figure in the chain of command had told us not to pay any attention to them.  I hope this post demonstrates that neither of those conditions were true and the entire team put forward their best efforts to ensure the safety of the crew and vehicle.

What Figure Did You Have In Mind?

“I can steal more money with a pencil than ten men with guns” – attributed to Al Capone’s bookkeeper

Whenever I did a press conference around a Space Shuttle event, there would always be one super hard question that made me stumble over the answer.  Technical subjects I had down cold, the hard ones were always about the cost.

How much did the new safety gizmo cost?  How much does ferrying the Shuttle across the country on the 747 cost?  How much does the launch delay cost?  What does each Shuttle launch cost?

If you look out on the internet there are ‘experts’ that will provide answers for these questions in plain and simple terms.  I would honestly answer: ‘it depends’.

The simplest way to calculate the cost of each shuttle mission was to take the annual appropriation from Congress, adjust for inflation over the 40-year history of the program, and divide by 135 (the number of missions flown).  Simple and totally inaccurate.  Why?  It’s complicated.

The Shuttle program manager was responsible for all the money spent but he could only control the portion called NOA – NASA Obligation Authority – which was a lot less than the money appropriated.  Each NASA Center had a ‘tax’ on every program inside their gate.  That is to say, if a program used a center, then the program contributed to the upkeep of the center:  paying the guards at the front gate, mowing the grass, paying the light bill.  Seem fair?

Well, if the Shuttle program was the ONLY program at a center – as it was for several of the largest NASA centers for a long time, the tax was eye wateringly high.  Does the VAB need a new coat of paint?  The Shuttle program gets to pay for that.  Does the MCC need a new roof? The Shuttle program gets to pay for that.  Does the A-2 Test Stand need a new flame bucket?  The Shuttle program gets to pay for that.  If the Shuttle program goes away, does the VAB still need paint and the MCC still need a roof?  Yes.  Paying for all those assets came to a head in about 2012 when the Shuttle program shut down and all the other programs had to scramble to find money to pay for infrastructure and center operating costs.

In that calculation of cost per launch, does one include those things that the agency still had to do whether or not the Shuttle flew?  It’s a judgement call depending on what point you want to make.

And how about civil servant salaries?  The Shuttle program was always in negotiation with the centers about how many civil servants to assign; but the program never got to vote on how many GS-13s vs GS-9s got assigned.  Generally, there were more civil servants assigned to the Shuttle program than the program really wanted.  Not negotiable, the agency needs to keep the ‘resource’ employed.  So, CS salaries were extracted from the program which had very little input into that calculation.  Carrying the salary costs so the agency had a ‘capability’ was not something that a Shuttle program manager really liked.

When one calculates the cost of ferrying the Shuttle from California to Florida, it makes sense to include the cost of the fuel used.  And one should probably include some allowance for maintenance on the 747.  But do you include an amount for the amortization of the capital expense of actually buying the plane in the first place?  And the cost of the modifications made to change it from a passenger plane into a shuttle ferry, a one-time expense paid years ago?  The pilots and the ground crews, do you include just the salary amount for the hours that they spend on the ferry flight, or include their entire salary for the year, maybe divided by the three-ish number of ferry flights performed? The generally stated cost of $1 million per ferry included all of that and more.

But if there were no ferry flights in a year, the shuttle program still paid for the pilots’ salaries and the maintenance of the airplanes – because the capability had to be there if needed.

Scrubbing a Shuttle launch was never fun, but they mostly saved the cryogenic fuel for the next launch attempt.  Some hydrogen and oxygen were lost due to boiloff, etc., but the cost of lost fuel was in the single digit thousands.  The work force was on payroll whether we launched or not. So, any calculation of cost for the scrub should really be based on the shift differential for the workers who were on the job overnight or on the weekend.  The generally stated $750,000 cost of a scrub included all the salaries – even though folks would be back at their desks working on the next launch if we hadn’t scrubbed.  So how accurate is that?

Is your head spinning?  I’m the son of a CPA that grew up with dinner table discussions about depreciation schedules and amortization costs and I still find it confusing.

The standing joke around the Shuttle program office in the last years goes like this: “The first Shuttle launch of the year costs $3 billion; all the rest of the flights are free.”

In other words, if there is to be a program at all, a specialized skilled workforce dedicated to that program must be paid, specialized facilities dedicated to that program must be maintained, and all of those things must be paid for, never mind however many times a year they are used.  A real space program is not a buy-it-by-the-yard kind of thing.  The incremental cost of any additional shuttle flight was more realistically in the neighborhood of $200 million – not cheap – but a lot less than the $1.5 billion figure that comes from the ‘simple’ computation that throws in everything and divides by 135.

At some point, the calculation depends on whether the calculator is selling or buying.  Does the author of the calculation want the program to look horribly expensive or reasonably cheap?

The title of this piece is the punch line to an old bookkeeper’s joke:

When asked ‘how much is 2 plus 2?’ the wily old accountant responded, ‘what figure did you have in mind’.

Any cost estimate depends on how it is calculated and a good question to ask is what is the motivation of the person doing the calculation.

Definition of Terms

I guess I’m getting to be an old curmudgeon (Hey Kids – get off my lawn!) but there are some irritants in life that just seem to capture my attention, no matter how trivial they may be.  So, if this post applies then use it; if you find it is off the wall please excuse it as an artifact of my advanced age.

 

A number of years ago my daughter, who was very interested in amateur astronomy, did a science fair project of the libration of the moon.  Here are a couple of pictures from her report:

Everybody knows that the moon shows only one face to the earth – we can never see the far side (please don’t call it the ‘dark side’).  It is “tidally locked” with the earth.  But maybe not quite.

It turns out that the moon’s orbit is not perfectly circular, there is some eccentricity in its monthly ellipse around the earth.  If you hear about ‘supermoons’ or ‘micro-moons’ you know that there is an apogee and perigee in the lunar orbit.  Not only does the moon come closer and farther away by a fraction every month but orbital mechanics dictates that it slows in orbital velocity at apogee and speeds up near perigee.  So, based on when you look it is possible to see a little of the ‘far side’ depending on where the moon is in its orbit.

Similarly, the moon does not orbit the earth at the equator, but its orbit is inclined about 5 degrees.  This means that sometimes the moon is a little north, and sometimes it is a little south of perfectly in line with an earth-based observer.

Put together, it is possible to see about 9% of the far side of the moon, in pieces, at various times.

The spot on the edge of the moon that is tilted the most toward an earthly observer is called ‘the libration point’.

Have you heard that term before?  I bet you have but in a different context.

Joseph-Louis Lagrange (1736-1813) was an Italian mathematician who played a large part in the development of the metric measurement system (SI) in post-revolutionary France.  He also studied orbital mechanics involving three bodies (e.g. sun/earth/moon) and mathematically proved there are locations around such an orbit which are gravitationally stable.  These points are called Lagrange points in his honor.  There are typically 5 such points and I will leave it to the student to research their locations.

As you can see Lagrange points and Libration points are quite different and literally have nothing to do with each other.

But if you read any number of popular media stories – and even several NASA technical papers – there appears to be confusion and the terms are used interchangeably.  This is so widespread that some dictionaries have started changing the definitions to keep up with what appears to be popular usage.

 

STOP THAT!

Unfortunately, the curmudgeon in me realizes that this erroneous usage has become so common that it will be hard to change usage in popular literature.

But at least you now know the difference.  And you, like me, will stop when you hear some ‘expert’ (never an astronomer) mixes the terms and think about how much ignorance is being displayed.

Oops

Thanks to Iron Flight for reminding me of this obscure situation.  And for Holly who wants to understand.

This mostly true story is about the Space Shuttle payload bay doors and what to do if they won’t shut properly.  And how we forgot about that emergency in our rush to achieve other mission objectives.

Some months ago, I visited Atlantis at the Kennedy Space Center visitor center.  Superbly displayed, it is the only shuttle vehicle with the payload bay doors open.  In the picture you can see some of the rube-goldberg mechanism that operates the doors critical latches to ensure the doors don’t spring open during aerodynamic flight.  A set of push rods and hinges at the sill open and close the doors.  A long set of push rod/bell crank mechanisms located in the front and rear edges of the doors operate the latch mechanisms.  On the bulkhead just aft of the crew cabin you can see the black cylindrical knobs which the latch mechanisms grab onto to secure the door in place.  More latches and their mechanisms ran down the centerline of the doors.  If this sounds complicated, it is.

Early in the shuttle days there was a huge concern about what to do if the doors did not close and latch properly at the end of the mission.  The doors had to be open during most of the orbit stay time for cooling and to allow satellite deployments or other objectives.  But during re-entry and the atmospheric part of the flight, the doors had to be firmly shut and latched down.  If they were to spring open and rip off, the vehicle would become uncontrollable and catastrophe would ensue.  Flight rules prohibited deorbit with any more than one set of latches not closed; and in that case the aerodynamic maneuvers were to be severely restricted.

Of course, the power to the motors that drove all this mechanism were redundant but that did nothing for a physical jam.  During STS-3 a thermal engineering test caused the doors to become banana shaped and prevented – for a short time – closing the doors.  This was done on purpose to see what the limits of the spacecraft were.  We found out.  And never did it again.

Long before STS-1 flew, a set of procedures for a spacewalking crewmember to deal with potential problems was developed and practiced.  I got my opportunity to take the class; use the tube cutters to cut a pushrod that had jammed, put clamps around unclosed latches to hold doors tight, and more.  There was a whole set of tools flown on every shuttle flight to deal with this contingency and every crew got at least one practice session on how to deal with it.

But the trick is that a space suited crewmember must get to the doors from the inside.  Normally this is not a problem; with an empty or near empty payload bay the EV crewmember just translates to the worksite; latches everything down, returns to the airlock door and ingresses.  Oh, and quickly gets out of the EVA suit and into the Launch/Reentry Suit and straps down because deorbit must occur shortly after the doors are closed and the radiators can no longer cool the ship.

If a payload blocked the path from the EVA worksite to the airlock, early in the program they were always jettisonable.  Not a problem.

Despite the complicated design, the payload bay door and latch mechanisms worked perfectly on every mission.  Over time, the concerns about having to deal with a failure faded away, even though the procedure and tools were on every flight.

During the evolution of the shuttle design, the European Space Agency built a laboratory to fit in the shuttle payload bay.  You can see the SpaceLab module on display right behind Discovery at the Smithsonian Udvar-Hazy center in Virginia.  Before the ISS, this was a facility to do micro-gravity research for up to three weeks in space.

But there is a trick that the designers missed.

If the mechanism on one of the aft payload bay door latches failed, and an EVA crewmember had to go back there to secure the doors down, there was no way for that crew member to squeeze past the SpaceLab module to get to the airlock with the payload bay doors closed.  Jettisoning the SpaceLab was not a task that an EVA crewmember could do.  The way home was going to be blocked.

Oops.

It may have been the first SpaceLab mission when we discovered this, but my recollection is that it was uncovered later, during a simulated mission.  Those darn Sim Sups!  They always made us work problems that were unrealistic!  Except that most of the time they weren’t.

What to do?  No more SpaceLab flights?  Unacceptable.

Review the risk?  Great history of reliability, low likelihood of needing to do the EVA.  Case closed?  Not exactly.

What if?

Here is the crazy resolution.  If the aft latches had to be secured on a SpaceLab flight, then the crewmember would just stay back there.

There was a good place to strap down at the bottom of the payload bay aft of the SpaceLab module.  It would be a short wait from finishing up with the doors until the deorbit burn, probably no more than an hour and a half.  From deorbit burn to the ground was about an hour and in normal flight the g loading was light, nothing more than 1.5g.  The EMU had plenty of consumables to work for the necessary time, but there are a couple of sticky things to resolve:

1.   The EMU uses a water sublimator to keep the crewmember cool.  As the payload bay repressurizes during entry, the sublimator will quit.  If the crewmember had selected full cool and chilled down as much as he/she could stand, it would probably be OK.
2. Getting out of the spacesuit: after landing, getting out of the helmet and gloves is not a problem.  Getting out of the hard upper torso by oneself is a chore but probably doable in 1 g if one is not in a hurry.  The lower pressure garment (pants) would not be a problem.
3. Getting the crewmember out of the payload bay, well that is a problem. Remember the doors are latched shut and clamps have been applied to keep them shut.  Surely the ground crew could figure something out . . . given several hours . . .  .

So that is the story.  Accept the risk because we think it is low; have a screwy contingency procedure ready if we’re wrong.

But that is not the way you really want to fly in space.

Night Flying

I just finished reading ‘Origins of 21^st Century Space Travel’ SP-2019-4415 from the NASA history office.  A fascinating book recounting the development of the Vision for Space Exploration in the Bush administration leading up to 2004.  Most of the action takes place in Washington, DC.  After all, this is about national space policy direction, not execution.  Those of us out in the field centers who were doing the detailed work to make the shuttle fly and build the ISS were not involved in the policy development.

My name appears in the book only on one of the last pages where the authors correctly observe that being able to launch the shuttle at night was critical to the ISS assembly.

This post is about how the night launch decision was made down at the program technical level.

Shortly after the loss of Columbia – while I was still assigned as the Shuttle Program Launch Integration Manager at KSC – the Eastern Range folks sent me a CD ROM with the radar tracking information from January 16, 2003.  Purportedly the radar tracking scans would show any debris falling away from the shuttle stack.  This was something that could be of great importance in the ongoing accident investigation.

I studied that data long and hard.

It was incomprehensible.

I consulted the experts who were experienced interpreting the squiggles and dots on the radar tracking data.  They confirmed there was a lot of things apparently coming off; none of it correlated to the time of interest that when the long-range tracking cameras saw foam coming off the ET.  No way to tell the nature of the material in those radar indications: foam, ice, metal; it was unknowable from the radar, just something that reflects radar waves came off.

A few things are known to came off during shuttle launches: ice from the extremely cold hydrogen channels on the main engines, engine covers from the booster separation motors.  And foam of course, but that is not very reflective.  And that was the extent of the list.  Nothing on the radar traces correlated.

Conclusion:  while the radar data was interesting and probably telling us something, it was not very useful given current state of understanding.

Fast forward two years to the hard work of getting ready to fly the shuttle after that accident.  One key element in our safety of flight rational was that detection of debris liberation during launch phase was mandatory.  Impact detection sensors we installed inside the leading edge of the obiter’s wings to detect anything that might impact those critical areas.   A bevy of new ground cameras with high magnification telescopes and super accurate tracking mounts were deployed around the launch area.  There were new in-flight cameras installed onboard both the SRBs and the ET.  A tremendous amount of money was spent to develop and install tracking cameras on NASA’s WB-57 aircraft to see the shuttle stack from a different viewpoint.

Trying out the new ground camera trackers

Two compact X-band radars were deployed on offshore ships to monitor for debris.  A powerful C-band radar became available from a Navy installation that was closing so we bought that and installed it well north of the launch pads to plug a hole in our radar coverage.  All these efforts were made at great taxpayer expense just to detect any debris that might come off the shuttle stack during launch.

Inspecting the X-band radar installation on one of the SRB recovery ships

Navy Surplus C-band radar installed north of Haulover Canal

 

 

 

 

 

 

 

 

 

Also developed was a special inspection boom that attached to the end of the shuttle robot arm.  The ISS crew was trained to use long lens cameras to photograph the underside of the orbiter as it did a backflip during near approach.

For the first two flights after Columbia, we required daylight launches to ensure that all those cameras would see anything that came off during ascent.  And we saw lots of stuff, mostly foam that missed the orbiter.  I wrote about that in How We Nearly Lost Discovery https://waynehale.wordpress.com/2012/04/18/how-we-nearly-lost-discovery/

 

During those first two flights we found out how well all the new detection/inspection systems worked.  The onboard cameras were fantastic; the new ground-based cameras were superb; the wing leading edge impact detectors gave a lot of false indications; the airborne cameras on the WB-57s just didn’t have the resolution to be helpful; and the radar was . . . incomprehensible.  Never could correlate any of the visually detected debris events to the squiggles and dots in the radar tracking data.  Seemed like we had wasted our effort and a lot of taxpayer money.  However, the real proof of health of the orbiter turned out to be in the in-space inspections; that saved the day.

To maintain our safety rationale in its entirety, many in the human spaceflight community really did not want to fly without that real time ascent monitoring of debris events, and the cameras were useless at night.  Orbital mechanics dictates the launch window to rendezvous with the ISS and without the capability to launch at night it might be months between available launch days.

What to do?  Rely on the radar, of course.  Not exactly an untruth but certainly a stretch.  Knowing that the real proof of safety was the in-space inspections, we continued to gather radar data and try to understand it.  Some experts even came to believe they could make use of the data.

I was never convinced.

Nonetheless, I publicly said that we could launch at night and would depend on the radars to detect any significant debris events during launch.

A fib?  A white lie?  An outright falsehood?  Never!  We poured over the radar plots every launch.

For what they were worth.

Calculating the Risk

Recently I have been involved in discussions about public risk resulting from commercial space flight.  It is interesting to think back to the shuttle days and how those calculations were made.

If you ever were invited to a shuttle launch and got a car pass to park on the causeway – seven miles from the launch pad – you might not have noticed the fine print at the bottom of the windshield placard – noting that you were accepting the risk to you and your family from launch hazards.  I’m betting you didn’t read that.  By the rules of the Range Safety organization, you should not have been there.  But the KSC center director had ‘accepted’ the responsibility for allowing you to be present when the risk calculations exceeded the standard that the Range set for the general public.

The limit for public risk was 30 chances in a million that one person could be seriously injured or killed from an event.  Don’t ask how they came up with that number.

The LCC building at KSC was constructed a full 3 statute miles from the Apollo launch pads because somebody did a calculation about the maximum explosion hazard from a Saturn V.  Three miles was far enough to be safe, or so they thought in the 1960s.

We still used that LCC building for the Space Shuttle launches and I was present in the firing room for several of them.  Every time before the clock counted down to zero, I would remember the hazard calculation for that room.

Follow this logic:  If, exactly at the moment of liftoff, the Orbiter blew up and took out not only itself and the astronauts but the External Tank as well so that the Solid Rocket Motors would be free flying, then if the hydraulic systems on the SRBs – now without any computer brain to command them – steered randomly but in just the right direction, and if the Flight Safety Officer took the maximum seven seconds to recognize the problem and send the destruct command, and then if the aft segment of the SRB – which did not have the linear shaped charge but was generally thought to peel apart when the rest of the booster destructed, rather stayed intact and burning – Then there was a non-zero probability that the flaming solid rocket aft segment could land on the LCC building with catastrophic results.

That is a lot of ‘ifs.’  And for several ‘ifs’ there was no way to calculate the probability, it is just assumed that they occur.  Given that all of those things happened, the only variable was the exact direction that the errant flaming segment might travel. Assuming an equal probability for each degree of heading, the ‘calculated’ hazard for the LCC exceeded the maximum allowed by the range rules, even for essential personnel.  But the firing room crew is essential to the launch so the KSC director again ‘accepted’ responsibility.

Those big louvers on the windows facing the launch pad were merely for shade, not for blast protection.  If something big hit them, they would probably just add to the shrapnel.

But even more, the KSC director ‘accepted’ responsibility for the hazard to workers in the Orbiter Processing facilities and the VAB who were not essential to the ongoing launch but were critical for maintaining schedule to the next shuttle launch.  And the KSC director also ‘accepted’ responsibility for the VIPs gathered on the balcony of the office building across the street from the LCC, and for the media at the Press site, and for other VIPs at the Banana Creek viewing stands.

That is a lot of ‘acceptance’ which let the Range organization off the hook, so to speak. Many of the guests and workers were not aware that they were in a hazard area.  It wasn’t a secret, it was quietly announced or in the fine print; never dwelt on.

That is for hazard areas as the Range calculated them.  Probably 1 chance in a ten thousand that you could be harmed.  Would you have still wanted to go?

As we go forward with ‘commercial’ launches, the FAA is responsible and has adopted virtually unchanged the calculations performed by the USAF Eastern Range.  But here is the catch; the KSC director is not allowed to ‘accept’ risk for commercial launches; its not in his/her area of responsibility.

Hopefully those smaller rockets will have a smaller hazard area than the shuttle.  But maybe not for some of the proposed super heavy boosters.

It will be interesting to see how all this plays out.  Public safety – what is your personal risk acceptance?  What probability would make you stay home and watch on TV rather than come see it live?  1 in a million, one in a hundred thousand, one in ten thousand, what?

And I wonder about the relatively unsophisticated calculation about the hazard from the Saturn V back in the 1960s.  I am guessing that most of the folks who came out to watch those launches would have accepted some risk, probably more than people would today.

They were willing to take more risks for a worthwhile cause in those days.

Finding Meaning in Apollo

I was born before Sputnik.  Yep, that old.

I grew up with the space age – the X planes, the Original 7, JPL sending Ranger to photo-bomb the Moon, JFK and LBJ, the whole enchilada.

I was crushed with Mariner 4 evaporated the Martian atmosphere into insignificance and when Mariner 2 turned the lush steamy jungle planet Venus into a sulfuric acid oven.  So many good science fiction stories were instantly pulped.

I watched Alan Shepard’s launch on a big black and white TV wheeled into our elementary school classroom.  The same for John Glenn.  When many of my friends kept baseball cards and knew the statistics on their favorite sports star, I kept track of astronaut flight assignments, and watched the Croft puppets do a real time simulation of Gemini spacewalks.

When Apollo 7 flew, I smuggled my transistor radio into middle school to listen to hourly updates on the flight from the radio news.  When she found the clandestine box, my teacher thought I was listening to the baseball playoffs like my classmates. But I wasn’t.

So, when it comes to the space, I got the bug early and hard.  The value of space exploration – robotic as well as human – is an article of faith for me; hardwired in from my earliest days.

Some years later, my college roommates could not believe that I spent three 7-hour days (during finals week!)  in the dorm TV lounge watching the Apollo 17 moonwalks live.  On the other hand, I couldn’t believe nobody else was watching with me.  (Yes, having to go to a special room where there was a TV was a thing then).

But a constant throughout those days was the criticism: ‘Why should we spend money on space?”  “We have problems enough here at home we should solve first!” “My taxes are too high, and this is just tomfoolery!” And after the first time:  “Been there, done that, why do it again?”

Maybe you thought that everybody was in favor of Apollo.  That was not the case, it was always controversial.

A couple of years later one historian offered this retrospective: “How different would the world have been if the Soviet Union had gone to the moon in 1970 and the biggest contribution the United States made to world affairs in that decade had been the war in Viet Nam?”  Yes, a very different world would have resulted; an alternate universe that should cause us all to shiver.

So, when I hear people question the spending proposals for renewed space exploration, I think ‘how old fashioned’ or maybe ‘how short sighted’.  Heard it all before; it was wrong then and it is wrong now.

I appreciated the entire summer of Apollo remembrances.  There were well-deserved tributes properly done.  If young folks think that everybody was in favor of Apollo, then we have not told the complete story.

And one last bittersweet thought:  When I came to work at NASA shortly before STS-1, space cadet that I was, I thought we would do this ‘shuttle’ thing for a couple of years, then assemble the space station as an embarkation point, and then head out for permanent outposts on the moon and to Mars and other places in the solar system.

Nope, I never expected to spend my entire professional career on the good old shuttle, with the ISS coming along right at the end.  How did we let that happen?

We need to keep that outpost on the frontier staffed and operating but more we need to take the next step.  Because I doubt if we ever have a 50-year celebration for the space shuttle – not like they will for the first boot print on the red planet.