Finding Meaning in Apollo

I was born before Sputnik.  Yep, that old.

I grew up with the space age – the X planes, the Original 7, JPL sending Ranger to photo-bomb the Moon, JFK and LBJ, the whole enchilada.

I was crushed with Mariner 4 evaporated the Martian atmosphere into insignificance and when Mariner 2 turned the lush steamy jungle planet Venus into a sulfuric acid oven.  So many good science fiction stories were instantly pulped.

I watched Alan Shepard’s launch on a big black and white TV wheeled into our elementary school classroom.  The same for John Glenn.  When many of my friends kept baseball cards and knew the statistics on their favorite sports star, I kept track of astronaut flight assignments, and watched the Croft puppets do a real time simulation of Gemini spacewalks.

When Apollo 7 flew, I smuggled my transistor radio into middle school to listen to hourly updates on the flight from the radio news.  When she found the clandestine box, my teacher thought I was listening to the baseball playoffs like my classmates. But I wasn’t.

So, when it comes to the space, I got the bug early and hard.  The value of space exploration – robotic as well as human – is an article of faith for me; hardwired in from my earliest days.

Some years later, my college roommates could not believe that I spent three 7-hour days (during finals week!)  in the dorm TV lounge watching the Apollo 17 moonwalks live.  On the other hand, I couldn’t believe nobody else was watching with me.  (Yes, having to go to a special room where there was a TV was a thing then).

But a constant throughout those days was the criticism: ‘Why should we spend money on space?”  “We have problems enough here at home we should solve first!” “My taxes are too high, and this is just tomfoolery!” And after the first time:  “Been there, done that, why do it again?”

Maybe you thought that everybody was in favor of Apollo.  That was not the case, it was always controversial.

A couple of years later one historian offered this retrospective: “How different would the world have been if the Soviet Union had gone to the moon in 1970 and the biggest contribution the United States made to world affairs in that decade had been the war in Viet Nam?”  Yes, a very different world would have resulted; an alternate universe that should cause us all to shiver.

So, when I hear people question the spending proposals for renewed space exploration, I think ‘how old fashioned’ or maybe ‘how short sighted’.  Heard it all before; it was wrong then and it is wrong now.

I appreciated the entire summer of Apollo remembrances.  There were well-deserved tributes properly done.  If young folks think that everybody was in favor of Apollo, then we have not told the complete story.

And one last bittersweet thought:  When I came to work at NASA shortly before STS-1, space cadet that I was, I thought we would do this ‘shuttle’ thing for a couple of years, then assemble the space station as an embarkation point, and then head out for permanent outposts on the moon and to Mars and other places in the solar system.

Nope, I never expected to spend my entire professional career on the good old shuttle, with the ISS coming along right at the end.  How did we let that happen?

We need to keep that outpost on the frontier staffed and operating but more we need to take the next step.  Because I doubt if we ever have a 50-year celebration for the space shuttle – not like they will for the first boot print on the red planet.

STS-121 The Hardest Launch Part 5: What Can We Learn?

Engineering is the art of modelling materials we do not wholly understand, into shapes we cannot precisely analyze, so as to withstand forces we cannot properly assess, in such a way that the public has no reason to suspect the extent of our ignorance.  – Dr A. R. Dykes

 A Flight Readiness Review is like Confession.  Every department must stand up in public and confess their sins, whereupon we all make a judgement about the severity and punishment.

Well, not exactly that but close enough.

At a NASA FRR each area is supposed to demonstrate that each and every piece part of a flight vehicle was designed with good engineering standards to accomplish the required purpose; and that further each part was manufactured to exacting tolerances and passed inspection; and still further that each part was tested or qualified to the conditions it would experience in spaceflight; and finally that all parts and all systems would work together to perform their functions successfully.  If the tiniest discrepancy was detected, an explanation must be forthcoming in excruciating detail with mountains of engineering rationale –  test and analysis – to show the discrepancy was acceptable.  A good FRR for the space shuttle could last two or more days with lengthy presentations, probing questions, sometimes acrimonious debate, and finally resolution:  fly or stand down and fix.

There was never a Flight Readiness Review without problem discussions.  I have found that every other space launch system which holds FRRs similarly discusses problems and discrepancies before deciding to fly.  These are complex systems built to exacting tolerance near the limit of what materials can stand.  There has never been a perfect rocket or a perfect launch; there are always discrepancies, problems, and issues.

Over the last several posts, I have recounted just a few of the major issues at the STS-121 FRR.  This is intended as an example and object lesson for those getting ready for future FRRs.  The STS-121 FRR was the most contentious that I ever experienced.  But it was like all those other reviews in that the imperfect system was laid bare, probed with great intensity, and at the end of the day a judgement made.  Those judgements were never based on perfect knowledge or understanding.  It is to be understood that perfect understanding – and perfect safety – does not exist.

For STS-121 the results were success.  But the questions remain:  were we smart or just lucky?  How close to the edge did we come?  Would it have been better to stand down and wait for better understanding or more perfection in the vehicle?  How long could we have waited before the program got cancelled?

It takes experience, skill, and understanding to come to a judgement in such matters; it is as much an art as a science.  How to know when enough has been done; how to know when it is safe enough; or how to decide to stop the clock and go back to work to understand and fix the problem.

At this point I generally quote Dr. Dianne Vaughn, a sociologist that studied how engineers at NASA make decisions in her book “The Challenger Launch Decision.”  Of course, it would take a sociologist to help us all understand the complex roles and interactions involved.  So here goes:

From Chapter 6 Engineering Culture:

1. “…the messy interior of engineering practice, which after the accident investigation looks like “an accident waiting to happen” is nothing more than “normal technology.” Normal technology…is unruly.”
2. “Judgments are always made under conditions of imperfect knowledge.”
3. “…experts are operating with far greater levels of ambiguity, needing to make uncertain judgments in less than clearly structured situations.”

Or just reference Dr. Dykes above.

The next time you hear a journalist reporting on a train crash or an aircraft accident or a ‘self-driving-car’ incident you can expect to hear it was ‘just an accident waiting to happen’ — because they ALL are.  The job of the engineers is to minimize the possibility of failure within the limits of what we know.

As we approach new space systems – especially launch systems – dissention should be expected at the Flight Readiness Review, some experts will believe there is more work to be done before the vehicle is ready to fly, that it is not safe enough.  That is because ALL flight vehicles are never perfect or perfectly understood.

Pray that there are leaders with judgement, maturity, and understanding that can properly evaluate the risks and make crucial decisions correctly.

When the countdown reaches zero say a little prayer and hold your breath.  Every single time.

Does that mean we should not go?

If you want to avoid all risk, you should stay in bed.

 “The desire for safety stands against every great and noble enterprise.” – Gaius Cornelius Tacitus (56-120 AD) 

 

STS-121 The Hardest Launch Part 4: Damage to the Heatshield

The ultimate controversy at the STS-121 Flight Readiness Review was the potential of fatal damage to the space shuttle during launch from foam loss from the External Tank.  The possibility of a replay of the loss of Columbia was the nightmare scenario that none of us wanted to experience.

In the first return to flight attempt, STS-114, events proved we had not adequately solved that problem.  Our initial conclusion that foam loss was due to installation error by the workers turned out to be incorrect.  Rather we found in that flight and the investigation that followed, that areas of foam on top of foam – where slightly different formulations of foam were used – would come apart due to something called ‘differential coefficient of thermal expansion’.  I wrote about that in depth earlier:  https://waynehale.wordpress.com/2012/04/18/how-we-nearly-lost-discovery/ .

The External Tank is a thin aluminum structure with surface parts sticking out; it all needs to be insulated because the liquids inside the tank are cryogenic.  Without insulation water from condensation out of the humid Florida air would quickly turn to ice.  Ice is an even bigger threat than loose foam.   So, insulation is required, and foam is the lightest weight type of insulation.  The vast expanse of tank surface area was covered by a formulation of insulation that was robotically sprayed on, but the small sticky-outie areas had to be covered by hand with different types of foam.  These ‘foam on foam’ areas proved to be the problem; they could come off during launch – particularly a from fittings called ‘ice-frost ramps’ (IFR).  We started an effort to shave down the foam as much as possible.  That limited the amount of possible debris.  But losses were still possible.

Ice-Frost Ramps of foam cover the attachment points for external lines on the outside of the ET

Using wind tunnels, super computers with Computational Fluid Dynamics programs, and finally Probabilistic Risk Analysis we studied the possible results.  Starting in January there was not much time.  In June, just before the Flight Readiness Review, the first results were finally ready to be revealed.  We needed to know what would be the likelihood of the shaved down foam coming off and critically damaging Discovery’s heat shield. Days before the FRR we got the results.

The results were not good.

NASA uses a matrix to plot the risks involved in any activity.  Five squares by five squares; rating risk probability from low to high and consequence from negligible to catastrophic.  The risk of foam coming off part of the External Tank and causing another catastrophe was in the top right-hand box:  5×5:  Probable and Catastrophic.  That square is colored red for a reason.

Completely unacceptable.  All the rules said no-go.

But the analysts did more than just present the results; they discussed the methodology used in the analysis.  In several critical areas they did not have the information they needed so they made worst case assumptions.  The results, they said, were probably overly bad; but they could not tell us by how much.  Weeks more of additional analysis might yield a more precise answer. And of course, we could return the vehicle from the launch pad back to the assembly areas where we could do more foam removal, if possible. Either way, it seemed like the launch would be delayed by a significant time.

This was a low point.  I felt, as the program manager who had multiple discussions with the analysts, that the results should have been better, in the area where the risk might be acceptable.  But it was hard to argue that the launch would be adequately safe.

Then up jumped the NASA Administrator, Mike Griffin, from his seat in the front row of the gallery, to the microphone.  What followed was a very interesting discussion of probability and statistics and a plausible interpretation that the PRA results were too conservative.  In fact, he concluded, the risk was acceptable.  Even the analysts present agreed with him.

What a turnaround.

With considerable discussion, and the notes penciled in on the Certification of Flight Readiness, the decision was to go ahead with the mission.

Were we smart or just lucky?  Nothing came off to damage Discovery on a beautiful July 4 launch.

As months past, and the analysis got better, the probability of catastrophe – as calculated – became acceptable, even low.

IFR Risk evolution

Reminds me somehow about what Mark Twain said about statistics.

Blood and Money

“The popular mind often pictures gigantic flying machines speeding across the Atlantic carrying innumerable passengers in a way analogous to our modern steam ships . . . it seems safe to say that such ideas are wholly visionary and even if the machine could get across with one or two passengers, the expense would be prohibitive to any but the capitalist who could use his own yacht.”   –W. H. Pickering, 1910

 

The news in the aerospace business is abuzz with discussion about the cost of spaceflight:  new studies on what Apollo actually cost back in the day, rumored estimates of the potential cost of the Artemis program through first landing, reports about cost overruns on some NASA activities.  All this stimulated my thoughts about costs and finance.

I need to hasten to add that I have not been privy to any inside information at NASA on these topics, although I believe they are working hard on budgets and cost estimates.  Rather I am thinking about history and how that informs the future.

Deep in the NASA mythology is a story about the legendary Administrator James Webb.  Allegedly, on the limo ride from NASA HQ to the White House to present the budget estimate for the proposed manned moon landing, Webb reviewed the $10 billion estimate and decided, on experience alone, that it should be doubled.  Moments later Webb presented JFK with a $20 billion estimate.  The actual cost came out to be about $24 billion or a 20% overrun from that estimate.

Myths – whether actually true or not – illustrate some value in a culture.  The value illustrated here is that all cost estimates come in lower than what the program will really cost.  Norman Augustine captured this in his book “Laws” about aerospace projects: (Law XXIV) “The most unsuccessful three years in the education of cost estimators appears to be fifth-grade arithmetic.”

Recent news media accounts of the Artemis program to potentially cost “$20 to $30 billion” over five years implies that sticker shock will occur in Congress.  This is a reasonable fear since sticker shock killed the Space Exploration Initiative in 1989.  And in 2009, Augustine himself concluded that the Constellation program was ‘unsustainable’ – code for Congress would balk at the cost.

I have not been briefed on any overall cost study results for Artemis.  My unscientific, gut response is surprise that the reported cost estimate isn’t twice as much.  Maybe the measures taken to build public-private partnerships and engage the efficiency of the commercial space industry allow for lower cost estimates.  We will see.

Either way, my judgement is that such a program would be a bargain basement good deal for the future of our country and humanity.  These numbers sound big to the average mortal, but as an investment in the future this is not much.  These figures are round off errors for the levels of funding for Medicare or the DoD budget; numbers Congress deals with regularly.

Enabling commerce in space will lead to unimaginable products in the future.  Could the Wright brothers have envisioned the ease with which so many people fly all around the world?  Could Alexander Graham Bell have envisioned an iPhone?  Space commerce today is a profitable with communications satellites, weather and resource monitoring.  Prospectively there could be return on investment – after the transportation is established – in off-earth mining.  But we cannot imagine what the real future most valuable space product will be.  It is as if we are trying to estimate the benefit of the transcontinental railroad in the 1850’s.  Leading the way to Mars – via the Moon – can lead to huge benefits to our descendants.

But I must say ‘Enough.’  Enough already with the fixation on financial cost, important as it may be.  Money is not the most important cost that the nation must face to commit to Artemis.  So far the discussion has not been about the real cost, the cost that might very well stop any advanced space exploration program:  casualties.

The Apollo program lost three crew in a ground accident; nearly lost three more on Apollo 13, and had more close calls than most people are aware of.  Just getting to low earth orbit, Soyuz lost two crews in two reentry accidents and had at least four more failed missions.  Shuttle, as we know all too well, flew 10 times as many flights as Apollo and lost two crews; 14 souls.

We must have a clear-eyed appreciation for the risk involved in space exploration.  Flying to the moon will not be much safer in 2024 than it was in 1969.  Exploration always comes with risk, and with some regularity exploration risk is realized.

The real cost of Artemis will be written in blood.  Face that fact.

This may be considered a poor time to bring this up – at a time when so many folks are actively working toward program approval.  Death is hardly a selling point.  But if we don’t recognize that fact, the program will come apart at the first bad day.

I for one think this an acceptable cost – if we are so unfortunate to incur it.  But the nation needs to make the commitment to put people in harms way.  This is only acceptable because of the benefits that will follow.  If we stop at the first accident because the public did not realize it might happen it would be better not to start at all.

Years ago, when I visited the Oregon trail, I was struck with a plaque that quoted one traveler: “Every few yards along the trail there was some jettisoned furniture or luggage; every quarter mile the carcass of a horse or oxen, and every mile a grave.”  As Winston Churchill once told the Canadian Parliament: “We have not journeyed all this way across the centuries, across the oceans, across the mountains, across the prairies, because we are made of sugar candy.”

We have a choice to make; we need to count the cost and the potential rewards.

Then we can commit.  Or stay home.

I vote to go.

Advising NASA

I’ve been passionate about space exploration from my earliest memories.  According to my mother, Sputnik was event that captured my imagination when I was 3.  Well before Apollo I had decided the only career for me was in the space program.  When I got my job offer from NASA just before college graduation, I was, well, over the moon.  Starting to work I was surrounded by a pantheon of heroes:  Chris Kraft, Gene Kranz, Max Faget, Glynn Lunney, and so many more.  And the astronauts:  John Young, Al Bean, Owen Garriott, Bob Crippen, Dick Truly.  I showed up a week before the Thirty-Five New Guys, you know:  Hoot, Sally, Judy, Dan, Dick, Guy, El, Pinky, and the rest.  My first boss was Steve Bales who saved Apollo 11; every office was populated with Apollo veterans. I felt sure that we would do this ‘shuttle’ thing for a couple of years, build the space station quickly, and head out to the Moon, Mars, and the rest of the solar system very shortly thereafter.

Real life did not turn out the way I had imagined it.  We had some great achievements in low earth orbit:  Hubble and Galileo, Compton and Chandra, SpaceLabs and SpaceHabs, culminating in building the ISS.  Somehow it turned into a full career spent exploiting near earth space, not exploring the vasty deep – unless you count all those telescopes.  Regrets; I have a few – as the song says – January 1986, February 2003.  At the end of my career, NASA wanted me to go to Headquarters in Washington.  Somehow, I knew that my engineering and operations skills would not be very helpful there.  Diplomacy was not my strong suit.  And I had family commitments that held me back from a move across the country.

So, I retired from government service and started a second career helping industry build and operate the complex devices required for spaceflight.  It allowed me to use all that engineering and operations knowledge and very little diplomacy was required.  Most of all, I could stay based near my family, even as I increased my frequent flier miles.

But it turns out that the US government wasn’t done with me.  Just over four years ago, Charley Bolden called me up and asked if I would serve on the NASA Advisory Council to help the agency chart the future.  I couldn’t turn him down.  Not sure we helped him much, but we did what we could.

The NASA Advisory Council – the NAC – is a Federal Advisory Committee subject to the Act of the same name – FACA.  By agreeing to serve, I became a Special Government Employee of NASA.  That’s right, you can look me up in the NASA directory just like the old days, but with my company phone number and email rather than the old one with @nasa.gov on the end.  It is a volunteer job for me; they pay my travel expenses, but my time is unreimbursed.  So, Charley, and now Jim, get my advice basically for free – before I retired, they paid me for my thoughts!  But three times a year I get to gather with the group of ‘graybeards’ at some NASA center (meetings rotate around), listen to all the interesting projects the agency is doing or planning to do, get a tour, and give our ‘advice’  in the form of observations, findings, and recommendations.  Not just human spaceflight but science, aeronautics, STEM support, and organizational topics.  So, my ‘pay’ is getting to hearing it all firsthand and seeing directly the work in progress.  My only failure so far is that I was unable to get a ride on the SOFIA flying telescope.  That would be really neat.  And maybe best of all I get to interact with old colleagues and some really exciting space celebrities.

A FACA committee must do its work in public, all in the open, discussion and debate, warts and all.  The agendas and minutes are published on the NASA web page and you can always dial in to hear the discussion (although the audio quality is not always good).  If you are local to the meeting you can attend in person.  There is always an open mike period scheduled for public comment.  It surprises me how little public involvement we gather.  I wish I knew how to encourage more people to come and/or comment.  Of course, the space beat reporters do a great job of covering the meetings and you can read about it in their articles, but you can get the whole story firsthand if you wish to devote the time to it.

As a Member at Large I got to kibitz on the subcommittee meetings.  I especially attended the subcommittee on Human Operations and Exploration because that is my special interest.  Ken Bowersox chaired that subcommittee – which like all the NAC subcommittees is made up of about a dozen retired experts in the field.  The subcommittee makes recommendations to the ‘big NAC’ for the agency but also to the Associate Administrator for HEO – Bill Gerstenmaier – directly.

Since Ken decided to go back to work for the agency, Jim Bridenstine asked me to take over the chair of the HEO committee.  Rats.  As a Member at Large, I could show up when I wanted to, partake in the debate, and leave.  As committee chair, I will have to lead the committee to some sort of consensus and provide a formal briefing and report to every NAC meeting; plus getting involved in the logistics of setting the meeting date, arranging for the location, building an agenda, etc.  Work!

Even worse, I really must think about the ethics conflicts now.  Earlier, the NAC was consumed by the long view and worked on policy directions such as whether NASA should send astronauts to the Moon first, Mars first, or maybe and asteroid first.  No conflict there any with my business interests.  As the lawyers would say, no ‘specific matters’ were discussed.  Now, with the new direction from the administration, I can see the NAC – and especially the HEO committee – being very involved with the next level of details.  And might very well impact some of my business interests.  So, in the last month – since my new appointment came out – I have been walking away from work.  Several major aerospace organizations – and some new starts – have asked for my paid consultancy on their near-term projects, many of which will compete for government contracts shortly.  I have had to turn them down.  No work for me that will provide even the appearance of impropriety.  My dilemma is where would my help be best – down and in helping the builders to succeed in design and execution – or up and out with the policy and strategy discussions at the NAC.  I’ve made the choice to serve on the NAC.

My goals for the HEO committee and for the NAC itself will be to listen thoroughly, research broadly, think clearly and give the best advice possible.  I would also like to work with the agency to make the HEO committee more diverse – not only in the usual sense of diversity but also more diverse in experience and opinion.

So, a long post and probably too much about myself.  If you have thoughts or advice for the agency – and I may regret this – please send them to me.  Many folks already do.  Please attend the HEO committee meetings whether in person or by audio conference – I will make sure you get access to the agenda and logistics.

NASA has an opportunity today.  There is a new push to go past low earth orbit, on to the Moon and Mars.  There is national leadership and opportunity.  As a nation we should not waste the opportunity.

I care because I really am passionate about space exploration.  Have been as long as I can remember.

STS-121: The Hardest Launch – Part 3 Wing Leading Edge

Returning to the history of the hardest shuttle launch I ever participated in, we earlier visited with the circumstances leading to the delayed launch of STS-121, the second test flight after the loss of Columbia and with the electrical sensor problems in the External Tank.  The next troubling problem addressed at the Flight Readiness Review was the integrity of the wing leading edge panels.

The very front part of the space shuttle wing gets incredibly hot during reentry, nearly 3,000 degrees F for almost half an hour.  A hole in the heat shield of the wing leading edge caused the loss of Columbia, so there is special emotion and attention focused on those panels.  Made of a special composite material, carbon phenolic cloth hand layered, impregnated with special resin, and fired under vacuum in a special oven, the reinforced carbon-carbon (RCC) panels were hand crafted with great difficulty.  And they were extremely hard to inspect.

Technicians in the Orbiter Processing Facility install wing leading edge RCC panels.

RCC panels showing the sharp bends on the sides and edges

 

 

 

 

 

 

Tests had been run showing that even very minor flaws in the interior of the RCC panels could burn through during reentry leading to the loss of another space shuttle and her crew.  The inspections of the wing leading edge consisted of a visual inspection with a magnifying aid and a ‘tap test’ where the technician tried to determine whether there was a void under the surface by rapping the RCC panel with his knuckles and listening to the sound.  These tests were not adequate to detect the kind of flaws that we now knew could be fatal.  A new inspection technique called ‘flash thermography’ used a strobe heat lamp to impart an energy pulse into the RCC panel and then an infrared camera monitored the temperature decay.  This could detect subsurface flaws in the RCC panels.  This inspection technique was new and there were no records of how the panels had appeared under flash thermography before STS-121.

It has been my experience that new tests often uncover things that were unexpected and not easily understood.  In this case the flash thermography test discovered ‘indications’ which might be a problem, or alternatively might be completely normal and not a problem at all.  In the corner areas of some wing leading edge panels, where there are folds or sharp corners, there were indications of unusual ‘signatures.’  In the worst case avoid under the surface could erode during entry.  Bad.  But the experts were divided.  In hand laying up the carbon phenolic cloth during manufacturing there could be wrinkles, especially in the complex geometry of a bend in the panel.  If these wrinkles existed at the manufacturing and had not caused a problem over many flights, we should be OK.  But if the signatures indicated a growing flaw that might get bigger every flight until the panel failed, that would be different.

We started lab tests to see if the difference could be understood.  Those would take time.  We reviewed the inventory; one spare set of panels was available, and we quickly moved to put the best spares on each orbiter in the locations where panels had the biggest ‘signatures’ – but not for Discovery, already on the launch pad when the discussion came to a head.  We ordered new panels to be made; but the factory throughput was 1 per month and each orbiter needs 44.  And the cost was high at $800,000 per panel.  That would be a long-term plan, not something in the short term.

The NASA Engineering and Safety Center was created after the Columbia accident to be an organization of the best of the best engineers who would be called in for the agency’s toughest problems.  The technical expert on the NESC was adamantly opposed to flying Discovery with any panels which were less than perfect.

At the Flight Readiness Review – almost a year after the last shuttle flight – with Discovery on the launch pad – we had the final discussion.  As Program Manager, I proposed flying as is.  My rationale was that the indications seen from this new test were likely (in my opinion) present from the manufacturing of the panels and had been through multiple reentries that were successful.  Secondly, most of the indications were in protected parts of the panels, covered by seals or other panels.  Thirdly, we were doing everything we could to replace panels as quickly as possible throughout the fleet – not an applicable argument for Discovery.  Fourth, we were moving quickly on testing to determine if ‘signatures’ as were seen on Discovery were a problem or not.  But those tests would take months to perform to gain sufficient sample runs to statistically prove the ‘signatures’ were not a problem.  I felt that the risk of not flying outweighed the risk of flying.  A programmatic stance, not an engineering one; if the second test flight returning after Columbia was delayed significantly, the pressure to end the entire program early would increase.  I admit that my recommendation was risky and not well grounded in engineering data.  But I had heard hours of presentations and discussions and that was my judgement.

Countering my position, the NESC pointed out that this was an indeterminate problem that could have fatal consequences and without more data it was an unacceptable risk.  Hard to argue with that.

The NASA Administrator was in the room and he stepped to the microphone and announced that he was accepting my recommendation and he would accept the risk.  It was very unusual, but that basically ended this topic.  The NESC does not sign the CoFR but if they had, I am sure they would have written a long dissent much as the Associate Administrator for Safety and the Chief Engineer did – but not for this topic, for the fourth one.  Stay tuned for the next installment.

So, what would you have done?  How would you have voted?  Stand down or go fly?  Acceptable risk or not?

We flew Discovery and it came home OK.  Months later we had the flash thermography tests showing that the ‘signatures’ had not grown in size with an additional reentry.  Months later, the laboratory tests demonstrated that RCC panels with fabric wrinkles deep inside were safe to fly.  But that is after the fact.  Sometimes decisions must be made under less than perfect circumstances.  That means somebody must accept the risk that things won’t go well.  It sounds easy until you put your signature on the line for it.

Miracle Workers

Scotty : Oh, you didn’t tell the captain how long it would really take, ya?  Oh, laddie. You’ve got a lot to learn if you want people to think of you as a miracle worker.

In July of 2005 we returned the shuttle to flight on STS-114.  I wrote about it earlier:  https://waynehale.wordpress.com/2012/04/18/how-we-nearly-lost-discovery/   We had made a terrible mistake that had to be rectified before the shuttle could be flown again.

At the end of August 2005, Hurricane Katrina slammed into New Orleans and nearly submerged the factory at Michoud – the only place in the world was that we could make or modify the External Tanks.  Even though the factory was saved, the nearly 2000 people that did the work to make those precision tanks lost their homes, their cars, and anything else that could not be evacuated in time.  It was weeks before we could even get in contact with all the evacuated workers and it would be months before their soggy houses could be made livable again.

Bill Parsons was the Program Manager and I was his deputy.  But Bill had close ties to the region and especially the hard-hit Stennis Space Center just a few miles again.  NASA tapped Bill to lead the recovery efforts for SSC and the other affected NASA facilities.  They promoted me to take his place in the program.

It was not long into September when headquarters started asking how soon we could get the Michoud factory going, fix the problems with the tank insulation in a definitive way, and when could they expect us to fly the shuttle again.

There were so many unknowns; first we had not yet begun to diagnose the foam problem or formulate a solution.   While the facility was largely undamaged, managing the human tragedy for the workforce was a huge question.  Infrastructure was in shambles; roads and bridges, electrical power and drinking water.  No grocery stores – or any other kind of store – was open anywhere nearby.  Having a dry, functional factory was worthless without people to operate it.

During the initial return to flight period when we sought to retrain the management to avoid poor communications and decisions, we hired Dr. Tufte to educate us on how better to communicate.  He gave (still does) a stirring indictment of the dangers of powerpoint presentations and highly recommended that writing white papers – requiring the use of complete sentences and organized paragraphs rather than bullet points – was superior.

So, wanting to demonstrate my new management tools, I gathered all the available information and wrote a white paper on the problems, potential solutions, and most critically about the schedule regarding flying again.  By shear dumb luck and overestimation of the problems my conclusion was that the shuttle would likely not fly before September 2006 – basically a year away.  Rather than having the typical NASA briefing, I mailed all the principal leaders – like the Administrator – my white paper.  Initially the response was gratifying; no significant questions, all issues and problems understood.  And while the gap in flights was disappointing, it was understandable, and we should proceed.

One of the most important qualities about NASA that you need to realize is that NASA cannot keep a secret.  Before we had any press releases or media briefings, like magic it was being reported that the shuttle would not fly before September of the following year.  Mike Griffin kept asking about the basis for the media reports.  The answer:  Wayne Hale’s white paper.  Somebody had passed it along.

Oh well.  The cat was out of the bag and even though we didn’t get to frame it as we would have liked, it was an accurate report.

Meanwhile, the real miracle occurred.  The dedicated workers down in New Orleans came back.  They came back to soggy houses that had to be gutted and rebuilt.  They came back to long commutes to the grocery store and any other kind of store.  They came back to the mess and in spite of it all, they came back to work.  Sooner and faster than anyone imagined that they would.  Lead by Wanda Sigur, the Lockheed-Martin workforce came back to work, and the factory came back on line months earlier than I had estimated.  In the meantime, John Muratore and his band of aerodynamics geniuses did the hard work to prove the fixes necessary to the ET would be safe.

Those are the real miracle workers, not us upper management dummies.

In the end, Discovery flew again on July 4; about two months faster than I had predicted.  It was hailed as a great success, both because we finally fixed the foam problem and because we flew sooner than we had predicted.

I learned a lot in those days.  Not the least of which was how to build a practical schedule to solve complex technical problems.

These days the space program is facing huge technical and schedule challenges.  There are all kinds of questions about deadlines and plans.  It seems to me that many of the schedules and launch dates that are published are invariably optimistic and don’t allow enough time to solve the inevitable problems that arise.

Sorta makes you want to give those folks the ‘Scotty’ advice.  Rather than building the quickest, shortest, most optimistic schedule, put some extra time on it.  Then, if you are good, or lucky – or perhaps a miracle occurs – everyone will be pleased.  The other way is not so pleasant.

When the Captain absolutely positively had to have warp drive back on line before death and destruction, Scotty always had a plan which got the job done – and sooner than required.   Somewhere there is a model to follow:  even with a demanding deadline, build a plan that can be done in less time than the deadline.

Just think about it.