Risk Management? Really?

Is the Emperor Naked? It Sure Looks That Way

The illusion here is based on the FAA’s belief that student pilots can be taught risk management skills. Learning to manage risk, however, requires prerequisite knowledge that student pilots typically do not have, nor is that knowledge typically available in any FAA document. So why does the FAA think that student pilots can be taught to manage risk? Why does the FAA sees clothes where I see a nudist? Let’s find out.

The dictionary defines risk as exposure to the chance of injury or loss. The key word here is chance. Risk management assumes that you know the chance—the probability—that an event might occur. For student pilots (or any low time pilot), event probability (risk) is extremely difficult (if not impossible) to evaluate, much less look up in some database that does not exist. It requires a background in statistical analysis or extensive aviation experience from which probability assessments might be intuited.

Keep in mind that NASA used risk assessment to identify the risks associated with flying the space shuttle. Their engineers, mathematicians and statisticians concluded that there was a 1 in 100 chance of losing a shuttle on every flight. Now that’s "authentic" risk assessment. Clearly this isn’t something that student pilots (or low-time pilots) are capable of doing. Most likely, these individuals don’t have the statistical knowledge to make these assessments (assuming the data are even available to use in making these assessments) nor do they have the working substitute for this knowledge known as “aviation experience” with which to make risk assessments. If you can’t assess the risk, you can’t manage the risk (unless you simply don’t fly... and what fun is that?).

Nevertheless, a fundamental feature of the FAA’s Risk Management Handbook is the Risk Assessment Matrix (shown below). This chart requires knowing the probability (i.e., likelihood) that an event might occur, and its severity should it occur. It also requires that you know what risk level is acceptable to you. Yet, the FAA never defines the "likelihood" values on the left vertical axis of their matrix. For instance, if the severity of an event is "catastrophic" but "improbable," it's listed in the green or "medium" risk category. So why is that airline pilots avoid areas of red on their radar screens (red areas have a 2% chance of destructive turbulence)? It seems as if a 2% chance of something happening falls close to the "improbable" range of likelihood, doesn't it? One might think so. Yet, airline pilots who see red colored radar returns on their radar screen avoid the area with a passion despite the event falling into the green or "medium" risk range on the FAA's matrix. So how can we talk about risk management when we can't speak with any precision about the risks we're willing to take?

 

 

The fact is that the FAA can't offer numerical "likelihood" values because they don't know what these values are for any given situation. Said another way, the FAA wants you to assess then manage risk, but they can't tell you what an acceptable risk is (outside one that falls into the right, bottom corner of the risk matrix). Can we really expect students (or low time pilots) to possess this knowledge and make these calculations? This seems unreasonable to me. Except for the extreme ranges of risk, the FAA's idea of risk assessment relies on vague impressions, incalculable assumptions and a great deal of wishful thinking.

The ironic thing, however, is that the FAA agrees with my assessment despite promoting the "fantasy" that risk management skill is within the skill range of a low-time aviator. Take a look at the following excerpt from Page 1-5 through 1-6 of its Risk Management Handbook (FAA-H-8083-2):

Managing Risks - Excerpt
"Risk is the degree of uncertainty. An examination of risk management yields many definitions, but it is a practical approach to managing uncertainty. Risk assessment is a quantitative value assigned to a task, action, or event.... Take the use of improper hardware on a homebuilt aircraft for construction. Although one can easily see both the hazard is high and the severity is extreme, it does take the person who is using those bolts to recognize the risk. Otherwise, as is in many cases, the [risk assessment matrix] chart is used after the fact.... Therefore, risk management is the method used to control, eliminate, or reduce the hazard within parameters of acceptability. Risk management is unique to each and every individual, since there are no two people exactly alike in skills, knowledge, training, and abilities. An acceptable level of risk to one pilot may not necessarily be the same to another pilot...."

ROD MACHADO'S INSTRUMENT PILOT'S AUDIOBOOK (MP3S ON DVD OR DOWNLOAD)On the other hand, what if we speak in terms of hazards instead of a risk? A hazard is defined as a danger or a peril. It’s something concrete that students can be taught to identify and avoid. Calculating probability is not required here because a hazard is not a data point on a risk assessment matrix. You don’t need to be a tai “chi-squared” master to recognize danger.

Hazard avoidance is precisely what good pilots do. They visually avoid thunderstorm cells by a minimum of 20 miles. They avoid flying into known icing conditions in an airplane not certified for flight into known icing conditions. Sure, they might climb or descend through an area of ice, but only if they know that they can quickly move into conditions not conducive to ice formation. They know their personal crosswind limits and avoid landing when those limits will be exceeded. There's no calculation or probability assessment taking place here (at least not as the FAA defines it). Good pilots think in terms of hazard avoidance; good instructors teach in terms of hazard avoidance.

Once again the FAA agrees with me on this point despite pretending that inexperienced pilots can actually assess risk. Take a look at the following excerpt from Page 17-4 of the FAA's Pilot Handbook of Aeronautical Knowledge (FAA-H-8083-25A).

Hazard and Risk - Excerpt
"Two defining elements of ADM are hazard and risk. Hazard is a real or perceived condition, event, or circumstance that a pilot encounters. When faced with a hazard, the pilot makes an assessment of that hazard based upon various factors. The pilot assigns a value to the potential impact of the hazard, which qualifies the pilot’s assessment of the hazard—risk. 

Therefore, risk is an assessment of the single or cumulative hazard facing a pilot; however, different pilots see hazards differently. For example, the pilot arrives to preflight and discovers a small, blunt type nick in the leading edge at the middle of the aircraft’s prop.... The seasoned pilot may see the nick as a low risk. He realizes this type of nick diffuses stress over a large area, is located in the strongest portion of the propeller, and based on experience, he doesn’t expect it to propagate a crack which can lead to high risk problems. He does not cancel his flight. The inexperienced pilot may see the nick as a high risk factor because he is unsure of the affect the nick will have on the prop’s operation and he has been told that damage to a prop could cause a catastrophic failure. This assessment leads him to cancel his flight." 

Student pilots are fully capable of avoiding in-flight hazards with a little training. All that’s needed is for their instructors to help them identify these hazards in specific and general terms.

Specific hazards are sometimes defined by the limitations placed on a student by the instructor or by the Federal Aviation Regulations. An instructor might inform his or her students not to fly when the direct crosswind component exceeds a specific value, when the flight visibility is less than three miles and when the flight cannot be made with visual reference to the surface, to name a few. Instructors can add as many additional items to the specific hazards list as necessary to ensure their student’s safety.

General hazards are best described in the succinct and condensed literary packages known as aviation aphorisms. These are wisdom-packed, hazard-avoidance lessons that students can easily remember. Here are just a few examples.

There’s nothing more useless than the altitude above you and the runway behind you. Takeoffs are optional but landings are mandatory. The only time you have too much fuel is when you’re on fire. The probability of survival is equal to the angle of arrival. Learn from the mistakes of others because you won’t live long enough to make all of them yourself. And so on.

These aphorisms are powerful behavior modifiers that can help student pilots make better decisions aloft and they have nothing to do with risk assessment. So fill your student’s noggin with these general, poetry-like quips of flying wisdom, along with your own specific warnings of hazards.

At this point you’re probably asking, What’s the downside to speaking in terms of risk management at the student pilot level? There are three important reasons not to pretend you’re teaching students to assess risk when they’re not.

First, students can’t manage risk unless they can assess it. Since they’re unlikely to have the skill to do this, they’re more likely to proffer a guess, instead. To the student pilot, risk assessment is guess assessment.

Second, an incorrect guess might result in the student making a poor decision while feeling confident of the outcome. There’s no upside to guessing wrong then feeling you’ve chosen wisely.

Third, while highly experienced pilots can assess probabilities based on their experience, they typically don’t do this. Instead, they reduce their mental workload by identifying and avoiding areas of known danger. As a general rule, very little (if any) probability calculation takes place inside a pilot’s noggin. When an airline captain sees red on her airborne radar screen, she’s not thinking, “Let’s see, red represents an area of 40 to 50 dBZs of radar reflectivity which is statistically associated with a 2% of destructive turbulence so I’m going to avoid that area.” Instead, she sees a hazard (a red colored echo) and avoids all contiguous echoes associated with that hazard by at least 20 nautical miles.

If you’re a flight instructor and want to offer a practical lesson for your students, teach them to avoid hazards. Don’t teach them something that’s not practical for them to do. Leave probability assessment for the professional mathematicians. Avoid the illusion; embrace the practical.

In the meantime, if you see nice looking outfit for sale, I know an emperor who needs one.

(This article originally appeared in AOPA Flight Training magazine.)

By Rod Machado | | CFI Resource Center, Learning to Fly | 3 comments
next post → ← previous post

Comments

  • Andy Foster - April 17, 2016

    Nobody’s arguing against teaching student pilots about risk and hazards; it is as essential as any other piece of the skill set we teach them. The discussion is about what approach is the most valid. Most people are overconfident in their assessments about likelihood and also consequence, including engineers and safety professionals who deal with risk on a day to day basis. What I believe Rod is pointing out ( and what I talk about more on my one blog) is whether a simpler approach of hazard avoidance is better. I certainly think it is, and as Rod points out, it’s what most pilots do.

  • Pat K - April 07, 2016

    Based off the learning principle of “what is leaned first is learned best,” it seems introducing the concept of risk management to a beginning student has value. Every flight has both common as well as varying hazards. Risk management is advocating that the student “investigates” the hazards prior each flight. Of course, the beginning student’s awareness of what are the risk is low. Each flight will subsequently build on that awareness. I am not aware of any data that supports a lower accident rate from initially teaching this method; however, I do not see any negative consequences resulting from it. When we start teaching a flight student, we don’t initially teach just stick then rudder. Similarly, we should not teach hazards then risk management. Like stick and rudder, they are fundamentally intertwined.

  • Andy Foster - March 17, 2016

    Rod,

    I fully agree with you on this. I’m a Light Sport CFI; but more importantly, I’ve worked in manned spaceflight safety for close to two decades (and most of my experience has been with the shuttle program) and had been wrestling with this issue for some time until I saw your article (You beat me to the punch writing about it!). This is ON TARGET! The process described was developed for managing risk at the program level, and as you pointed out, the judgement of likelihood and consequence both must be rigorous (and even NASA programs struggle with it at times) and is best served when supported by “quantitative” analysis (like PRA). At the pilot level, especially the student pilot level, it’s a lot more practical and wise to simply avoid any risks you can and go home to fly another day.

Leave a comment

Stay in Touch

Physical Product Ordering Only (800) 437-7080

If you'd like to order a PHYSICAL product by phone, please call the number above. Digital (downloadable) products can't be ordered by phone.

Latest Posts

  • Rod's Letter to the FAA and ACS Committee Members

    Yes, really! The Private Pilot ACS adds 363 Risk Management items to the PPL practical exam testing requirements. Students are responsible for knowing the answers to each and every one of these items, all of which require highly subjective answers (according... read more

  • Minimum Cost Private Pilot Certificate

    Greetings Folks:Below is a recent letter from a young man named Joel Thomas. Joel earned his pilot certificate at a very low cost using many of the recommendations I've made over the years. Yes, it's entirely possible to earn a... read more

  • Why Vx and Vy Change With Altitude

    Recently, someone asked about why Vx and Vy change with altitude. This isn't necessarily an easy thing to understand since it involves several variables. So here's a modified "quick" version of the explanation on this topic that is covered in... read more

  • How to Flare Any Airplane Any Time and Anywhere

    Princess Buttercup and I were walking on the Redondo Beach pier last month and unknowingly stumbled onto the “live” movie set of Big Momma 2. As I passed one of the props, an ice cream kiosk, I stopped to buy... read more

  • The Airman Certification Standards

    As most readers of my blog know, I'm not a fan of the new Airman Certification Standards (ACS) for many reasons, not the least of which I originally posted with the FAA in 2013 (click here to read that response). Apparently, the... read more

  • Risk Management? Really?

    Is the Emperor Naked? It Sure Looks That Way The illusion here is based on the FAA’s belief that student pilots can be taught risk management skills. Learning to manage risk, however, requires prerequisite knowledge that student pilots typically do... read more

  • Hot Props

    Crocodiles and Propellers - Not Much Difference Sometimes we need to be reminded just how dangerous a propeller can be.  The first video below shows a fellow pulling a propeller through to check for nicks when the engine started and... read more

  • Striving for Too Much, Too Soon - Fantasy Flight Training

    Early in the previous decade, the FAA began heavily promoting a flight training concept known as scenario-based training (SBT). SBT was billed as a highly structured script of real-world experiences to address aviation training objectives in an operational environment. Hailed... read more

  • Rod Machado's "FREE" Private Pilot Flight Training Syllabus

    I'd like to offer you two different syllabi (FREE). The first is my Private Pilot Flight Training Syllabus. This is intended for use by the flight instructor as well as the student. Both should have a copy for their own... read more

  • Flight Instructor Training Resources

    I'm often asked about resources for CFI applicants. Here are a few resources that you might find useful. When I discover more, I'll be happy to list them here.  Rod Machado’s Private Pilot Handbook - A  must have book to help... read more

  • In Defense of Stick and Rudder Training

    It happened in the early 1990s. That was the time we saw the diminishing influence of WWII flight instructors (and their instructional progeny). Our pilots didn’t fly jets during that war. Instead, they flew airplanes that demanded exceptional stick and... read more

  • Bad Instructors: How They Behave and Why You Should Know

    I'm curious about your experience with bad flight instructors and hope you'll add to this blog with  your comments. Of course, most flight instructors do a fantastic job. I just can't say that LOUD enough: Most flight instructors do a fantastic... read more