A Not-So-Simple Truth

When I was a rookie Flight Controller at JSC’s Mission Control, my boss had a hand drawn poster on the wall of his office.  During tedious checklist and flight rule reviews I had plenty of time to study that poster; I wish I had a copy of it today.  It featured a hand with the thumb upraised and words that went something like this:  “Thumb’s Rule:  A Simple, Easy to Understand Falsehood is More Useful than A Complex and Incomprehensible Truth”.

In the Internet age, it seems many folks practice this maxim.  After all, the 10 second sound bite on the evening news or the 140 character tweet conveys all the information the public needs to know about the complex and difficult problems facing us today, right?  Just like the shouting heads on the “news” channels have replaced thoughtful adult conversation about how to move forward through the challenges of 21st century life.

Nor is a thousand word blog post going to fully convey the complexity of the real world.

February is a month for introspection for me, and the events of 8 years ago have been on my mind.

In the Columbia accident investigation report, there are several pages devoted to the use of a computer program called “Crater” which analyzed potential damage to the thermal tiles.  The results provided from that computer program indicated that no serious damage had been done to Columbia’s tiles and therefore a safe landing would occur.

Disaster occurred instead.

The accident investigation board spent a substantial amount of time looking into the Crater program and castigated NASA for using Crater to assess damage on Columbia.  That program was developed for other uses and had been validated to use for tile damage assessment only with very small impacting objects.   Clearly it was used inappropriately during the last flight of Columbia.

The Columbia Accident Investigation Board got all the big things right in my opinion, and most of the small things as well.  But on this particular item, well, I’m having second thoughts.

Suitably chastised, we wrote new rules governing the use of any computer program analyzing information for the space shuttle.  The limits of the validation of each program had to be clearly stated and the results should not be used if the inputs were outside the validation testing.

Tile Damage From Impact Test

Meanwhile, we spent thenext three years shooting different objects at shuttle tiles to determine how they could be damaged by impacts.  This work became very important to the assessment of safety from liberation of debris in areas we could not completely eliminate it.

And, after all this testing, it seemed that Crater accurately predicted tile damage at least to a first order effect. Even for larger, harder, faster debris.

 

 

Really.

So the simple statement that NASA used the Crater program outside its validated limits and came to erroneous conclusions is . . . not quite the complete truth.

So what really happened?

It is complex.

During Columbia’s last flight, significant effort was made to assess the possible damage from debris during launch.  Ultimately, it was misdirected effort.  From the fuzzy long range video the strike had been near the front of the left wing; but whether it was on the fragile tiles or the RCC wing leading edge panels was unclear.

During the flight, the RCC experts were unanimous in the opinion that the RCC would not have been damaged from such a strike.  One of them, an engineer for whom I have great respect, told me after one of the MMT meetings: “That RCC is tough.  We shot ice and other hard stuff at it and couldn’t break it.”  This was the consensus of opinion in late January 2003.

Unfortunately, as we know, that was wrong.  During the investigation, it took a full scale test which shot a big hunk of foam at a flight-like installation of RCC panels to prove to the experts that damage could occur.

No damage evaluation, no computer models, no assessment other than expert opinion was used during the flight of Columbia to come to the conclusion that the RCC was undamaged.

So what conclusions can we draw in retrospect?

Should computer models only be used within the limits to which they are validated by test data?  That would be a good practice if analysis tools could cover all conceivable situations.  But the real world always surprises us.  It is important to know when a situation has left the limits for the validated used of a predictive program.  But knowing that, sometimes those predicted results can be useful and sometimes they can even be correct.  Use that analysis with care and great trepidation, but sometimes it is better than nothing.

A more important lesson is that experts, even in their field of expertise, can be wrong.  As managers of complex vehicles flying in extreme environments using exotic technology with small margins for safety, we tend to trust our experts.  That trust must be tempered with the knowledge that they are not always right.  Somewhere in the budget limited, schedule compressed environment that is modern spaceflight there has to be enough capacity, enough capability, and enough time/money/equipment to check on the experts’ opinion.

Especially when it means life or death.

That is a complex, almost incomprehensible truth that is going to haunt spacecraft designers and operators for a long time to come.

I’d replace that poster in my old boss’s office with this new rule of thumb:

“You Are Not As Smart As You Think You Are”

About waynehale

Wayne Hale is retired from NASA after 32 years. In his career he was the Space Shuttle Program Manager or Deputy for 5 years, a Space Shuttle Flight Director for 40 missions, and is currently a consultant and full time grandpa. He is available for speaking engagements through Special Aerospace Services.
This entry was posted in Uncategorized and tagged , , , , , . Bookmark the permalink.

74 Responses to A Not-So-Simple Truth

  1. Beth Webber says:

    From your opening paragraph to your final conclusion I am rocked by the simple truth of all you have said. What dismays me the most about the end of the Shuttle, with no clear program in site, is the loss of all those so-hard-to-learn lessons in the wake of the Colombia tragedy.

    I hate to think of the high price we will pay to relearn them.

    Beth

  2. P. Savio says:

    “Foam can’t do that”…..

    • waynehale says:

      It all sounds so easy and straightforward in documentaries like this. The world is complex, choices are gray not black and white, and the truth sometimes takes a long time in revealing itself. Too easy for folks after the fact to say that they would have done things differently.

  3. Fascinating blog as ever. As many have said, The shuttle is still an experimental vehicle. The sad part is with 131 successful flights behind NASA they just seem to be understanding the shuttle and the envrionment it operates in. The people, procedures, experience base, etc is just starting to become mature and it’s all about to end.

    I have to agree with Beth’s comment about re-learning things.

  4. Dave H. says:

    Wayne,

    Thank you for writing this as we near the conclusion of the STS program.
    The weekend after the accident, I was at my mother’s with a wheelbarrow tire in a sink full of dishwashing suds to try and find a leak. Some NASA spokesperson analogized the foam strike to being hit by an empty Styrofoam cooler blown out of the back of a pickup truck at 55 miles per hour!

    How this person chose that analogy just stunned me, because I have been on the receiving end of one of those coolers while riding my motorcycle. It shattered on my windshield and sent the bike into a brief-but-intense “tank slapper”!

    As Dr. Osheroff so eloquently noted, normalization of deviance and complacency were major contributors to the accident.

    The guys from “Mythbusters” probably loved Scott Hubbard’s foam launcher, which dispelled all doubts after blowing through the RCC and damaging the instruments and cameras behind it. It was almost like standing in the forward compartments of Titanic as the iceberg sliced through them. Unless you wove tungsten wire through the RCC like you see in vandal-resistant glass, RCC is just as brittle as any other carbon. Anyone ever swing a baseball bat at it?

    Thankfully, these pre-Columbia attitudes did not prevail in the runup to last year’s attempted launch of Discovery, as a stand-down and complete inspection of the ET stringers revealed that 94 out of 108 required reinforcing…and that it would have taken the failure of only four adjacent stringers to cause the tank to fail in a tragic fashion upon liftoff.

    They might not have even made it past the top of the tower…

    And again, in deciding to inspect both Endeavour’s and Atlantis’ tanks “just in case”.

    Computer models are only as good as their creators’ knowledge base and the data which is input into them. To be fair, NASA used its experience with foam shedding and impacts because it had happened during every launch since Day One with no ill effects. No one ever tried to improve the foam’s integrity either…why should we waste time and money on a non-problem?
    Then, one day…

    Today, we watch as the operators of those nuclear power plants in Japan which survived the earthquake but not the tsunami that followed struggle with consequences of a scenario that no one obviously believed would ever happen. The power plant I work in has 125 volt DC batteries too, but if you submerge the batteries (they’re always on the ground floor because they’re very heavy) submerge the motors and submerge the switchgear, you’ve got a real problem.

    No, we’re not as smart as we think we are, but if you remember that Murphy was an optimist, at least you’ll have a leg up.

    It’s been a long road, Wayne, getting from where things were eight years ago to where they are today. Two more flights to go, no time for slacking off yet.

  5. Dave says:

    Wayne,
    This story reminds me of a recent conversation I had regarding a much less consequential situation…. The bottom line was: a shitty plan is better than no plan at all. Unfortunately for the crew of Columbia, there was little that could practically be done at the time other than come in and hope for the best. It didn’t work out, but we learned how to improve on open analysis and debate of flight risk. It doesn’t look like that lesson counts for much in the current zeal to pursue “more cost-effective” commercial access to space.

    • waynehale says:

      I completely disagree with your comment on numerous levels.

      • Mark Bray says:

        I disagree with it as well Wayne.

        However, I take issue with your article. The CAIB was wrong in determining that CRATER was valid or invalid. The problem was not technical – it was cultural.

        The Subsystem Manager for the RCC hardware stated clearly that we can analyze every scenario in the book, but until we know the nature of the damage, we can not know whether the analysis is valid. We could have taken a picture from the ground to determine the location and severity of the damage. NASA elected not to do so. Having known this and run proper analysis, we may have elected to attempt heroic measures and possibly save the crew. At least we could have tried.

        I do not agree that this was a complex problem. What made for a bad day was the lack of desire to confront the problem. I am not saying I would have done any better or differently – that is not the point. I am saying that the lesson NASA should have learned (and the lesson I learned through that catastrophe) is not a technical one – but one of leadership and culture. So long as we are driven by annual (and now bi-weekly) budget cycles, I do not see how NASA can affect the internal change necessary. Not having a vision for the future makes it practically impossible.

        There is a stark difference between the NASA of old and the NASA today. When people say they want Apollo back, they are not referring to the vehicle or the astronauts – they are saying they want the NASA culture of old to be resurrected.

      • waynehale says:

        Your comment completely confused me. I don’t get your point.

      • Dave says:

        A frank and direct response… Thanks! I respect you and I would appreciate an education… What would be the (practical) plan had tile damage been confirmed prior to deorbit? What lesson(s) is it the commercial sector taking from Columbia or NASA’s manned space experience in general?

      • waynehale says:

        The commercial sector is taking quite a lot from Columbia and NASA’s experience – at least as far as I have anything to do with it.

      • Dave says:

        Wayne,
        I am confident you are a very positive influence on commercial space… I am hoping there are going to be more like you contributing.

  6. Dan Cordes says:

    I have always wondered, if the damage was know before reentry, what would have been done to bring the astronauts home safely? At the time there was no way to repair the TPS and no way to launch another shuttles to rescue them in time. So, the real question is, “What would/could NASA have done?”

  7. Gary Miles says:

    Dr. Hale, were there options available to repair the shuttle if an assessment had been made that the tiles were damage and the shuttle could not land safely. The impression in the media was that there were little to no options for rescuing the shuttle in that scenario. Could such limited options been part of the underlying reasons for going with the experts that there was no tile damage and the shuttle could land safely?

    • waynehale says:

      I’m no doctor.
      No, no tile repair capability existed at the time of STS-107. There is good capability now.

      • Gary Miles says:

        So why not have the STS-107 shuttle inspected while it was berthed with ISS or before that? Why not confirm the initial assessment with actual empirical evidence? Did not think you were a shrink. Just wanted to understand the feelings, or perhaps motivations, behind the decisions concerning the Columbia accident. As I have often experienced in life, I sometimes missed important details or little signs that something is about to happen. Our brains are conditioned to look at patterns and when something fall outside those patterns, we tend to reject or ignore it.

      • Gary Miles says:

        My apologies, STS-107 was not an ISS mission. But here is a quote that came from a Space.com article and can be found in many other media articles from that time as well:


        While Columbia was still in orbit, some engineers suspected damage, but NASA managers limited the investigation, on the grounds that little could be done even if problems were found.

      • Gary Miles says:

        I have been reading the CAIB report and Chapter 6 makes it appear that some of the assessments regarding Columbia’s foam strike had more to do with the launch schedule and budget impact more than the actual threat to the safety of the shuttle. There are references to Linda Ham particularly on page 138. Any comments?

      • waynehale says:

        Had I been the chair of the MMT I would have made the same decisions that LInda did. She has been unfairly vilified.

  8. Achim Vollhardt says:

    A true expert will be the first one to question his own judgment when there is the slightest doubt.

    Mr. Hale, if there were more experts with your expertise and work philosophy in the decision making levels for the things to come after the Shuttle program, I would feel much more confident about the future than I currently do.

    • waynehale says:

      You do an injustice. I was just as much a part of the Columbia decision making as anybody. We were all wrong. I am no better than anybody else who was involved. Don’t put your confidence in some guy on a white horse who has all the answers. Do good work. Ask tough questions. Don’t be intimidated. Bring data and good analysis, not conjecture, opinion, or rumor.

  9. Andrew W says:

    Thanks Wayne,
    I think often when the experts get it wrong it’s when they start to rely on human intuition or so called common sense rather than the physics and maths. Humans aren’t very good at intuitively assessing the physics of events that are outside normal human experience.

  10. Shaun Standley says:

    One of the good developments to come out of CAIB and the subsequent lessons learned activity was the new Standard for Models and Simulations (7009) and the associated guidebook, currently in draft form. These address questions such as use within or outside a models’ validated range, and overall they are a thoughtful and useful approach to the complex subject of model validation.

    • waynehale says:

      In draft form? After 8 years?!? No we wrote guidelines immediately. They were approved at the Shuttle PRCB. You must be talking about some sort of multiprogram NASA document. Surely it hasn’t taken 8 years to write that!

  11. Miles G says:

    “the RCC experts were unanimous in the opinion that the RCC would not have been damaged from such a strike. One of them, an engineer for whom I have great respect, told me after one of the MMT meetings: “That RCC is tough. ”

    My understanding is that there were no RCC experts left. The subsystem management system had been shut down some years before and the only engineer left who had any involvement in the original Shuttle thermal protection system testing and certification had expertise on tiles but not on RCC, and that his comment when questioned during the Columbia mission, was that ‘he thought the RCC was much tougher than the tiles’.

    • waynehale says:

      Not at all true. I talked with several folks in the engineering organization who had responsibility for the RCC from earliest days and were continuing in that work. The title of subsystem manager had been transferred to other folks but the expertise was still there and still engaged.

      • Karen says:

        But I believe the expert you mention above, the engineer for whom you have great respect, was not actually an expert in RCC. One of the problems we run into is assigning authority to people who SPEAK with authority, even if they are only speaking from their own sense of self confidence without really having spent significant time working the details of the issues. As Bob Ryan says the people you have in the trenches, working the hardware, they are your experts. Not the guy who’s two levels above them with 30 years of experience in Program reviews who knows how to speak assertively.

      • waynehale says:

        You must have the wrong guy. I talked to the guy in the trenches.

  12. Dave H. says:

    “You do an injustice. I was just as much a part of the Columbia decision making as anybody. We were all wrong. I am no better than anybody else who was involved. Don’t put your confidence in some guy on a white horse who has all the answers. Do good work. Ask tough questions. Don’t be intimidated. Bring data and good analysis, not conjecture, opinion, or rumor.”

    Wayne, you are still so hurt inside by this, and I wish that there was something anyone could do for you as well as everyone else.
    Think of Linda Ham…she has to forever live with the “what if” created by her refusal to permit high-definition photography to be taken, dismissing the engineers’ concerns with “I don’t think there is much we can do.”
    I forgive her, knowing what I’ve learned in the past eight years.
    What was your specific part in this panoply…what would you do differently if you suddenly found yourself back in 2003?

    How would you get anyone to believe you? Where was your precious “data” to dispute what the TPS engineers already knew, that foam impacts happened all of the time and there had never been a problem before?

    What if I told you a story?

    Have you ever seen “Phenomenon”?
    What if George Malley’s friend Professor Ringold was a CAIB member, and together they worked very hard over the past eight years to make sure that everyone who goes to work in space came home again safely? Would the miracle be that the only thing George ever asked for was an opportunity to contribute something worthwhile?
    Would the true miracle be that NASA set aside its trepidations and actually used George’s ideas…and that they all worked?

    Would knowing that a lot of people out here not only forgave you but were standing behind you as well ease your pain?

    Maybe I’m reading more into your thoughts than you intend, but Apollo 1, Challenger, and Columbia have left deep scars across a lot of people’s lives…you can still see the hurt in Gene Kranz’s face 44 years later.

    But sometimes life is not about your failures, but how you recover and learn from them.

    • waynehale says:

      Real life is not like the movies.
      Pop psychology is not helpful.

      • Dave H. says:

        “Real life is not like the movies.”

        Would you then say that the movie “Apollo 13” is a work of fiction? Using Vulcan logic, could one logically conclude that if movies were based upon real events one could therefore conclude that real life formed the basis for movies, which are merely stories told in visual form?

        “Pop psychology is not helpful.”

        I fix machines, not humans. Since you like Star Trek movies, and frequently quote from them, our exchange here has reminded me of the exchange between Spock’s half-brother Sybok and Captain Kirk, and what Kirk said to Bones about it. To quote:

        “Damn it, Bones, you’re a doctor. You know that pain and guilt can’t be taken away with a wave of a magic wand. They’re the things we carry with us, the things that make us who we are. If we lose them, we lose ourselves. I don’t want my pain taken away! I need my pain!”

        Perhaps, like Kirk, you need your pain…and perhaps I should mind my own business and stop trying to help others.
        I certainly do not mean to upset or offend you.

  13. Kenneth Chang says:

    Wayne writes, “The results provided from that computer program indicated that no serious damage had been done to Columbia’s tiles and therefore a safe landing would occur.”

    But that’s not true. I’ll have to go dig up the Boeing PowerPoints, but Crater predicted a long gash in the underside that would go deeper than the thickness of the thermal tiles. The last thing in the PowerPoint trail was that a thermal analysis would be performed. I’ve never seen anything that shows the results of that thermal analysis (or even confirmation it was performed).

    Then engineers said Crater was conservative, the “densified” layer at the base of the tiles would survive, and everything would be fine.

    Thus, it seems the opposite happened. Crater predicted catastrophic damage to the tiles. But because it was simplistic and the event was out of the validated range, the engineers dismissed it and relied on their intuition instead. It was this jump in logic — as far as know, not backed with reassuring data — that always struck me as the biggest misstep in the chain of events leading up to Columbia’s destruction.

    • waynehale says:

      The conclusion was that the accident was caused by failure of the RCC leading edge panel, not failure of the tiles.

      • Kenneth Chang says:

        Right. I was just disputing your assertion that Crater “indicated that no serious damage had been done to Columbia’s tiles.”

        To the contrary, slide 5 of the Jan. 23, 2003 presentation by the Boeing team is titled, “Damage Results from ‘Crater’ Equations Show Significant Tile Damage.”

        The first bullet point on that slide states, “‘Crater’ indicates that multiple tiles would be taken down to the densified layer.”

        The Boeing team calculated a number of impact parameters. The best case was a gash 19 inches long, 2.4 inches wide and 2.6 inches deep (for tiles 2.8-3.1 inches thick).

        The worst case was a gash 31.9 inches long, 7.2 inches wide and 2.8 inches deep for tiles 2.3-2.4 inches thick. Which is to say, the tiles would be scraped off entirely.

        I think you would agree that these Crater predictions were not saying “no serious damage.” Even though it turned out the impact was on the leading edge and not the tiles, my point is that it wasn’t Crater that gave the shuttle team a false sense of confidence.

        On page 10 of the presentation, the Boeing team presented thermal predictions for a number of different scenarios for damage to the heat shield. For four scenarios where a single tile was lost or there was just superficial damage to the RCC coating, they concluded there was no issue.

        Case 5 and Case 6 are “Lower Wing Area, (32 x 7.2 x 2.8) Damage” and “Main Landing Gear Door (several tiles lost)” The corresponding boxes under “Results” are blank. And yet the conclusion on slide 13 is “safe return indicated even with significant tile damage.”

        As I said, it seems the engineers performed a diligent analysis using the tools (Crater) that they had, and tools predicted a gash up to three feet long going down to the skin. I’m guessing that prediction, taken at face value, would not have guaranteed a safe return. But it seems someone decided, “Nah, that can’t be,” the thermal analysis was never completed, and the issue was closed.

        Just finishing that last step — the thermal analysis — would have confirmed there was potentially a major issue, and then maybe events would unfolded differently.

        Thank you for writing this wonderful, informative and insightful blog.

      • waynehale says:

        I think we stray a bit far from the point of my post, which was directed at whether or not to trust experts.

  14. mike card says:

    Wayne, although your “You are not as good as you think you are” rule is a life’s lesson, sometimes one must just say damn the rules and make decisions based on one’s gut. Even so, the outcomes are problematic.

    • waynehale says:

      Is making a decision based on your gut better than getting expert advice?

      • mike card says:

        NO, if expert opinion is available. Sometimes the expert advice is unavailable, or may be questionable, one must use his best judgement. For example, a single-handed sailor at sea with no communications capability has no source of advice except his own experience.
        Mike Card

      • waynehale says:

        So when experts are available, and their advice seems firm with high confidence we should always take it; right?

      • Andrew W says:

        Wayne said: “So when experts are available, and their advice seems firm with high confidence we should always take it; right?”

        Good question. For everyday life if there’s doubt we should look at getting a second opinion.
        Perhaps a good leader in the situation being discussed would have found the time to ask “why?”, eg: As an expert, why have you concluded that there’s no great likelihood of the foam impact having caused major damage to the Shuttle’s insulation?
        Sometimes experts need to be grilled to make sure they’re not pulling the wool over the/their eyes.

      • Dave says:

        It depends on what my gut says about the “expert”.

      • waynehale says:

        I completely concur. But be careful; especially question those experts who give you the answer that you would like to hear!

      • Andrew W says:

        Is “gut” another word for intuition? Because my main concern about trusting the opinion of experts is the worry that they’re relying on their gut/intuition rather than the sweat and technical approach that makes them the experts.

  15. Steve Pemberton says:

    Flying in space is orders of magnitude more complex than flying an airliner (which itself is very complex). And yet there is a remarkable record that isn’t talked about much. In fifty years of U.S. spaceflight, not once has there been a fatal accident caused by someone making a mistake. By mistake I mean forgetting to tighten a bolt, or forgetting to flip a switch, or neglecting to check something, or miscalculating something, or giving someone the wrong instructions. It is almost beyond comprehension to think of the thousands upon thousands of critical tasks that had to be done right on each and every one of those flights. I’m sure there were some close calls, but not once in fifty years did anyone make a mistake that led to disaster. Compared to any other human endeavor, this is a record that is perhaps without equal. And yet it seems that for the most part this achievement by NASA and its workers goes unrecognized.

    Programmatic mistakes did lead to two Space Shuttle disasters. Even in the can-do spaceflight business it seems that sometimes limits have been accepted. Like shedding foam. Imagine being told that you have to apply foam onto a fuel tank, and that after being applied the foam is going to sit around for several months, including being out in the elements for several weeks. Then on launch day the foam will be subjected to cryogenic temperatures and flexing of the structure that it is attached to. This is followed by two minutes of horrific vibration, during which there is transition back to temperatures above cryogenic and more flexing. For the grand finale the foam has to endure supersonic air loads with resulting high temperatures. And oh by the way, during all of this not one piece of foam is to ever, ever come off. Makes my head spin just thinking about it. No surprise that the answer was “can’t be done, you’re going to occasionally lose foam”. And “if damage did occur to the Orbiter, there’s nothing you could do about it”. Uncharacteristic perhaps for a NASA that is normally known for doing the impossible, but perfectly normal in human terms.

    The loss of Columbia seemed to create a new resolve within NASA to do the impossible. Return to Flight began a new era where NASA pushed the previously perceived practical limits of safety, thoroughness and rigor to new heights. Return to Flight has been a remarkable achievement, and it seems that even in these final flights they aren’t satisfied yet and continue to push towards improvement. I think that there are many lessons to be learned, not just from the failures, but also from the successes.

    • Steve Pemberton says:

      In case it wasn’t clear, my statement “not once in fifty years did anyone make a mistake that led to disaster” was referring to in-flight fatalities. There were of course several accidents on the ground including the Apollo 1 fire in 1967 and the asphyxiation deaths of two Rockwell workers on the launch pad in 1981.

    • waynehale says:

      I thought programmatic mistakes were made by people; that makes them mistakes caused by somebody. Your comment is nonsequitur.

      • Steve Pemberton says:

        Sorry about that, I was trying to make the distinction (perhaps not very clearly) between task oriented mistakes which can be caused by things like miscommunication, memory lapse or unexpected distraction, and strategic mistakes which are more likely judgmental in nature, obvious only in hindsight in many cases, and caused by a variety of complex factors as illustrated in your article.

        I was also trying to express a couple of other points. But whereas you do a great job of coherently linking together multiple topics in a single message (like your recent post “Breaking Through”), I am realizing that without this skill most of us are better off sticking to a single point in our comments. Even better of course is if that point is related to the current topic.

  16. David S. Schuman says:

    Thank you, Mr. Hale. I most admire your sense of humility in the face of engineering challenges.

  17. Chris Johnston says:

    Looking at this from the outside it seems like there was another, unseen hand working here. The RCC on Columbia had been flown before – a lot, in fact – and it seems likely from the people that I’ve talked to that there were unseen and possibly unknown damage mechanisms working on it over its history. If the bulk of the testing that gave the experts the confidence in its strength was done on virgin RCC they may have been dealing with a significantly different material than what was found on Columbia on the day of the accident. It does not change the outcome or the conclusions of the post-accident testing (which I thought was done on more pristine material) but is does caution that the real world seems to find ways of breaking things in ways that nobody anticipated.

    • waynehale says:

      This was a very popular theory in the early days following the loss of Columbia. Exhaustive impact testing with both new and flown RCC has conclusively shown that there was no significant strength loss from multiple flights. The theory was disproven.

  18. Patrick says:

    Just wondering why my post is still in moderation… is it not a valid question? Was I misinformed?

    Respectfully,
    Patrick

  19. Gary Miles says:

    Given the fact that foam from the ET had broken off and struck the shuttle on a number of previous missions and damage to TPS was found following the conclusion of some of these missions in post-flight reviews, were there not proposals to examine the shuttle in flight with an extra space walk in case of a foam strike? NASA may have conducted lab experiments to analyze the extent to which the TPS and RCC could be damaged by falling. However, I would think that empirical observations of the real world phenomenon inflight to check the ground assessments would be a priority. Simply because no catastrophic failure occurred on the previous missions where foam strikes were observed is not guarantee against castastrophic failure. If there were such proposals, then why were they not approved?

    • Gary Miles says:

      I omitted a word. I meant to say falling ‘foam’ in previous comment.

    • waynehale says:

      EVA is a tremendous risk. Perhaps the long string of successes has caused you to normalize that risk, but it is huge. The benefit of an EVA must always be balanced by the huge risk that it entails to the crew members. EVA is not undertaken for light reasons. Prior to STS-107, all foam loss damage had been very minor. Only in retrospect would anybody suggest that a crew member from STS-107 should have gone EVA to inspect the wing. Oh, and the fact that they could not get to the area that we now know is damaged is a real complicating factor. No RMS, no handholds, no “jet backpack” on that flight.

      You see, it is easy to be an arm chair Monday morning quarter back. Its a lot harder to make decisions when you know the risk on the other side and the future is . . . unknown.

      Nonetheless, you have drawn us away from the central theme of the blog post . . .when do you trust experts . . . and when do you not trust them . . .

  20. Kim Curry says:

    Thank you for posting, Wayne.

    January is my month for reflection, remembrance, and re-commitment. I have to admit my own blog has been pretty silent lately. Sometimes I struggle to find the right words.

    When do we trust the experts? When multiple independent analyses point to the same or similar conclusions. When independent models corroborate each other.

    Unfortunately, science can take YEARS to develop a consensus. And some consensuses are later disproven (like hormone replacement therapy for heart health).

    The real-time environment can’t wait years for an answer. What do you think of the 80/20 guideline?

  21. Steve Pemberton says:

    I found it interesting to think of how to apply your advice about computer models to the central topic of trusting the opinion of experts. Perhaps the same advice works for that as well, i.e. the importance of knowing when a situation has left the limits for the expert, and yet still taking their advice (with care and trepidation) if it’s all that you have at the moment.

  22. David says:

    I have a quote in my office that reads: “Murphy had it wrong: Everything that can go wrong usually goes right, and then we draw the wrong conclusions.”

    This is from someone named Langeweiche.

  23. Rodney Rocha says:

    I was on the STS-107 Debris Assessment Team (DAT), which consisted of over 30 engineers from NASA civil service and contractors from United Space Alliance, Boeing, and Lockheed-Martin (Michoud Assembly Facility). There were also NASA safety engineers on the DAT. The entire DAT membership unanimously wanted an extra photo of Orbiter Columbia in order to anchor and initialize its analysis assessment with real and unequivocal data. Several on the DAT communicated this urgent message upward through various NASA organizations and managers. The obstacles to these efforts and the attendant communication breakdowns are well documented in Chapter 6 of the Columbia Accident Investigation Report and other first-hand published accounts. As Wayne has stated, this was a complex issue involving technical and human factors.

  24. Dave H. says:

    ” When do you trust experts . . . and when do you not trust them?”

    It all boils down to your personal knowledge of the subject matter and your personal comfort zone.

    Staying within the realm of the events leading up to the loss of Columbia, a casual observer would certainly have to trust the evaluations of the experts in the TPS field simply because most of us don’t work in that arena and therefore wouldn’t expect to have a knowledge base that would permit us to challenge their work.

    “Is making a decision based on your gut better than getting expert advice?”

    Wayne, I have recorded a TV documentary wherin a few NASA engineers wished that they’d stood up during the flight…”you see, it is easy to be an arm chair Monday morning quarter back”…especially after most of the pieces have been picked up.

    Those engineers saw a huge chunk of foam slam into the RCC…they did the math…but all of the existing data contradicted their data. The crossroads came about five days into the mission…at the morning briefing Linda Ham stood up and said that she’d heard rumors that some of the engineers wanted to have high-resolution photographs taken. They’d twisted arms and called in favors behind the scenes, but all they needed was a blessing.

    They didn’t get it, because their “gut feeling” wasn’t substantiated by the historical data, and as we all know, in God we trust, all others bring data.

    That smug, self-righteous anachronism needed to go away. Better to consider input from as many sources as possible and winnow the chaff from the wheat.

    But then again, sometimes a “gut feeling” can be far better than data, depending on how much you trust the person telling you that something’s wrong but they have no data to back their feeling up with.

    It took Scott Hubbard to validate the “gut feeling” of the engineers who asked for the photography.

    You want the ultimate Monday-morning quarterback scenario?
    What if Linda Ham had said “I don’t think there is much we can do, but have the photos taken and report back on what you find.”

    Argue with me and tell me what that would have hurt? Go ahead, I’ve cried enough over this in the past eight years, and will be damn glad when Atlantis’ wheels stop rolling because I can have my life back!
    I do have to admit that being the only family from West Mifflin to walk around beneath Endeavour was something my mother took to her grave with a smile.

    “Real life is not like the movies.”
    I had that exact same perspective at 8:59 AM ET, Saturday, February 1, 2003.
    Words I’ve eaten since then?
    “And this affects me how…?”

    Here’s an exchange from “Yesterday’s Enterprise” that is germane to this conversation:

    Capt. Picard: You must have some idea how things have changed.
    Guinan: I look at things, I look at people, and… they just don’t feel right.
    Capt. Picard: What things? What people?
    Guinan: You. Your uniform, the bridge…
    Capt. Picard: What’s the matter with the bridge?
    Guinan: It’s not right!
    Capt. Picard: It’s the same bridge. Nothing has changed.
    Guinan: I know that. I also know it’s wrong.
    Capt. Picard: [sighs] What else?
    Guinan: Families. There should be children on this ship.
    Capt. Picard: What? Children on the Enterprise? Guinan, we’re at war!
    Guinan: No we’re not! At least we’re not… supposed to be. This is not a ship of war. This is a ship of peace.
    Capt. Picard: [ponders this] What you’re suggesting…
    Guinan: I’m not suggesting. That ship from the past is not supposed to be here. It’s got to go back.
    Capt. Picard: Who is to say that this history is any less proper than the other?
    Guinan: I suppose I am.
    Capt. Picard: Not good enough, damn it, not good enough! I will not ask them to die!
    Guinan: Forty billion people have already died! This war’s not supposed to be happening! You’ve got to send those people back to correct this.
    Capt. Picard: And what is to guarantee that if they go back they will succeed? Every instinct is telling me this is wrong, it is dangerous, it is futile!
    Guinan: We’ve known each other a long time. You have never known me to impose myself on anyone, or take a stance based on trivial or whimsical perceptions. This timeline must not be allowed to continue. Now, I’ve told you what you must do. You have only your trust in me to help you decide to do it.

    • waynehale says:

      Last evening the local TV station rerunning Star Trek TOS showed the episode where Kirk was split in two: Good Kirk, Bad Kirk. When I was a teenager watching the show that seemed so profound and important; now it seems a little trite and not quite so deep. But as much as I like Star Trek, enough already. Lets move on to real life. Your understanding of the motivations of both Linda Ham and Scott Hubbard is not profound.

      • Dave H. says:

        “Lets move on to real life. Your understanding of the motivations of both Linda Ham and Scott Hubbard is not profound.”

        I never said that it was “profound”. Both of them had jobs to do at different times, and both did the best they could with the information they had to work with.
        There is no logic in postulating “what if” until eternity; this is not a class in temporal mechanics.

        And until the wheels stop rolling and Atlantis and her crew are safely home, the jury is still out on whether NASA was able to learn anything from Columbia.

        If I may say something here without you taking offense, many of your replies here appear to be similar to those of a college professor who doesn’t understand why his students aren’t getting his lessons.

        For what it’s worth…

      • waynehale says:

        I’m no college professor. Sorry if I offended with my brief replies. I do get a little peeved at folks who – without much study or insight – give glib explanations for complex issues.

  25. Dave H. says:

    Truth be told…here you ARE the professor! You alone have the experience, exclusive knowledge, insights, and behind-the-scenes stories to share with us.
    Speaking for myself, and probably others, we certainly learn a lot from the things you share with us.

    The losses of Challenger and Columbia are certainly emotional for you, and it takes a lot to overcome your emotions and share the things that you have. For that, I take my hat off to you.

    There is a saying that you can’t learn anything while your mouth is moving. I beg to differ with that, because if you never speak up you’ll never learn if you have a valid point of view or not. You’ll never find out if the things you believe in are wrong or not. That said, those of us who were not part of your experiences will only know as much as we watch on TV, read in books or magazines, or research ourselves here on the Internet.

    When we give those seemingly glib explanations for complex issues, we’re exposing the limits of our knowledge.
    We simply don’t know, and sometimes, we don’t know what we don’t know.

    But we do appreciate your patience with us.
    Thank you.

  26. allenjrichardson says:

    Wayne:

    On March 12, 2011 you wrote about the Columbia accident and – “a computer program called “Crater” which analyzed potential damage to the thermal tiles. The results provided from that computer program indicated that no serious damage had been done to Columbia’s tiles therefore a safe landing would occur. Disaster occurred instead,” Your statement about the “Crater” prediction is not true. As documented in the Boeing Assessment briefing dated January 23, 2003 (which received wide distribution within Shuttle engineering ) the Crater program predicted clearly that the potential damage to the tiles was far too severe for safe landing of the vehicle.

    My Boeing colleagues and I derived the analysis tool that later became the Crater Program early in the Shuttle program. Accordingly. I was a consultant to Boeing during the Columbia investigation and was privy to flight details. My primary task was to assist in determination of the cause of the accident. This turned out to be foam impact on the wing leading edge and my impact analysis of this event (presented in a paper to the AIAA) predicted it to be the cause and this was later confirmed by test. A secondary task was to assist in determining the cause of the incorrect “safe landing” prediction. The assessment briefing clearly presents the crater predicted damage as a long gouge several inches wide completely through the tiles and spanning several of the 6 inch tiles. Detailed thermal analysis and test over the years had firmly established that the maximum tolerable impact damage to the tiles was limited to damage to a single tile. Therefore, Crater predicted an unsafe entry.The cause of the incorrect “safe landing” call was the result of a decision by both NASA and Boeing management to override the Crater prediction.

    The Boeing assessment briefing speaks for itself in regard to the fatal damage prediction by Crater and is a must read for anyone interested in the details of the Columbia disaster. I will email you a copy on request. I also have a more detailed piece which elaborates on the role of Crater which I would be happy to pass along.

    The legacy of many prominent NASA and Boeing employees involved in the Crater decision would benefit greatly from your claim that the erroneous “safe landing ” prediction was due the Crater Program, particularly if that claim is incorporated into your upcoming book. It is my hope that honesty will prevail and that your misinformation about Crater in the Not So Simple Truth article is corrected.

    Allen J. Richardson (714) 526 1703

    • waynehale says:

      I keep learning more and more about this subject. What I heard at the MMTs during STS-107 was not the entire story, obviously. What is interesting to me is that Crater was basically validated after Columbia.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s