Boeing Failed To Consider Pilot Workload When It Designed and Tested The 737 MAX

The U.S. investigators of the 737 MAX incidents released their first recommendations for required changes on Boeing’s best selling airplane. Their analysis of the accident causes confirms our earlier take.

A recent New York Times Magazine piece by one William Langewiesche blamed the pilots for the crash of two 737 MAX airplanes. We strongly criticized that Boeing friendly propaganda piece:

The author’s “blame the pilots” attitude is well expressed in this paragraph:

Critics have since loudly blamed it for the difficulty in countering the MCAS when the system receives false indications of a stall. But the truth is that the MCAS is easy to counter — just flip the famous switches to kill it. Furthermore, when you have a maintenance log that shows the replacement of an angle-of-attack sensor two days before and then you have an associated stick shaker rattling away while the other stick shaker remains quiet, you do not need an idiot light to tell you what is going on. At any rate, the recognition of an angle-of-attack disagreement — however pilots do or do not come to it — has no bearing on this accident, so we will move on.

An AoA sensor failure and a following MCAS incident will cause all of the following: an unexpected autopilot disconnect, an airspeed warning, an altitude disagree warning, a stall warning and, after MCAS intervenes, also an over-speed warning. The control column rattles, a loud clacker goes off, several lights blink or go red, several flight instruments suddenly show crazy values. All this in a critical flight phase immediately after the start when the workload is already high.

It is this multitude of warnings, which each can have multiple causes, that startle a pilot and make it impossible to diagnose and correct within the 10 seconds that MCAS runs. To claim that “MCAS is easy to counter” is a gross misjudgment of a pilot’s workload in such a critical situation.

Cockpit Confidential: ... Smith, Patrick Best Price: $4.42 Buy New $10.00 (as of 08:40 UTC - Details) After that piece was published Langewiesche went on CNBC where he repeated his slanderous allegations:

“It amounted to just a runaway trim”
“There was never a reason to ground [the MAX]”
“[Boeing’s] largest mistake was to overestimate the quality of the pilots it was selling its airplane to”

Last week the National Transport Safety Board (NTSB) released 13 pages long recommendation (pdf) resulting from its investigation into the 737 MAX incidents. It strongly supports our view and counters Langewiesche’s claims:

[T]he MCAS becomes active when the airplane’s AOA exceeds a certain threshold. Thus, these erroneous AOA sensor inputs resulted in the MCAS activating on the accident flights and providing the automatic AND stabilizer trim inputs. The erroneous high AOA sensor input that caused the MCAS activation also caused several other alerts and indications for the flight crews. The stick shaker activated on both accident flights and the previous Lion Air flight. In addition, IAS DISAGREE and ALT DISAGREE alerts occurred on all three flights. Also, the Ethiopian Airlines flight crew received Master Caution alert. Further, after the flaps were fully retracted, the unintended AND stabilizer inputs required the pilots to apply additional force to the columns to maintain the airplane’s climb attitude.Multiple alerts and indications can increase pilots’ workload, and the combination of the alerts and indications did not trigger the accident pilots to immediately perform the runaway stabilizer procedure during the initial automatic AND stabilizer trim input.

The pilots did no do wrong. There were multiple alarms that required their attention. Boeing’s assumptions that the pilots would immediately recognize a runaway stabilizer and react appropriately turned out to be wrong:

Although the NTSB’s work in this area is ongoing, based on preliminary information, we are concerned that the accident pilot responses to the unintended MCAS operation were not consistent with the underlying assumptions about pilot recognition and response that Boeing used, based on FAA guidance, for flight control system functional hazard assessments, including for MCAS, as part of the 737 MAX design.

It wasn’t the pilots who failed. The system was designed in a way that made it extremely difficult if not impossible for the pilots to handle it in the available time.

Boeing never analyzed or tested the complete chain of events that would follow from a failure of an Angle of Attack sensor. Boeing tested an MCAS failure but only as an isolated incident, not as it would happen in real life:

To perform these simulator tests, Boeing induced a stabilizer trim input that would simulate the stabilizer moving at a rate and duration consistent with the MCAS function. Using this method to induce the hazard resulted in the following: motion of the stabilizer trim wheel, increased column forces, and indication that the airplane was moving nose down. Boeing indicated to the NTSB that this evaluation was focused on the pilot response to uncommanded MCAS operation, regardless of underlying cause. Thus, the specific failure modes that could lead to uncommanded MCAS activation (such as an erroneous high AOA input to the MCAS) were not simulated as part of these functional hazard assessment validation tests. As a result, additional flight deck effects (such as IAS DISAGREE and ALT DISAGREE alerts and stick shaker activation) resulting from the same underlying failure (for example, erroneous AOA) were not simulated and were not in the stabilizer trim safety assessment report reviewed by the NTSB.

An AoA failure triggers a number of alerts and the pilots need time to sort those out. An MCAS failure does not leave time to sort out anything. The pilots must react immediately. But they can not do so when multiple other alarms caused by the AoA failure also demand their attention.

Boeing built the MAX as the fourth generation of a plane that was designed in the 1960s. In each generation new systems and alarms were added and certified. But each added system was only tested in isolation. New fault tree analysis for the plane as a whole was not required as the original certification of the first 737 was still accepted as a base. No simulator tests were done that tested the ability of pilots to cope with multiple alarms that happen when a defect causes multiple interdependent systems or instruments to fail. Had Boeing made realistic assumptions about a pilot’s reaction time to multiple alarms MCAS would have had to be implemented differently.

The NTSB recommends that Boeing and other manufacturers make new system safety assessments that consider the effects of all possible flight deck alarms and indications on the pilots reaction time when they respond to the failure of flight control systems. It asks for design changes of the alarm systems and for additional training. The NTSB recommends that the Federal Aviation Administration and other regulators include those demands into their general rules for aircraft certification.

The NTSB recommendations will likely induce the FAA to require additional changes on the currently grounded 737 MAX. They also seem to to push the FAA to require additional pilot training.

The NTSB report is bad news for Boeing. Most competing airplanes are much newer than the 737 and have multiple electronic sensors that can be easily combined to sort through and prioritize alarms. The 737 MAX is still largely based on the old mechanical and electrical systems of its predecessors. That makes it difficult to add a system that coordinates and prioritizes the cascade of alarms that can happen during certain events. The required changes will come on top of other changes that international regulators have loudly demanded.

It will likely take several more months until the 737 MAX is again certified and can go back into the air. Boeing still produces 42 MAX per month. It will now likely have to stop the production line until sound solutions for all the open questions are ready to be implemented.

Reprinted with permission from Moon of Alabama.