Watch-out for those “self” driving cars.

Cadillac says self driving cars are the way of the future and that may be but if the recent crashes of two Boeing jets, due to what certainly appears to be because of failures in their Maneuvering Characteristics Augmentation System (MCAS), are an indication of how self-driving cars may behave, Cadillac and all auto makers may want to “proceed with caution”. It is and they should.

The Wall Street Journal has reported that preliminary findings from officials investigating the Ethiopian Airlines Boeing 737 MAX 8 crash suggest that a flight-control feature automatically activated before the plane nose-dived into the ground based on data retrieved from Flight 302’s black boxes.

The MCAS is a system that automatically lowers the nose of an airplane when it receives information from its external angle of attack (AOA) sensors that the aircraft is flying too slowly or too steeply, and at risk of stalling. The MCAS has three basic components integral to all automated machines.

• Sensors — are devices that collect data by taking measurements of the environment in which a machine is operating. They are analogous to the five senses all human beings use to measute our environment. In the case of the 737 Max 8 the principle sensors collect data needed to measure it’s AOA.
• Code — processes the data, measures/calculates the AOA, collected by the sensors and sends signals to the mechanics of the machine to behave accordingly, like lower the tail of the airplane, forcing the nose downward. The code is also analogous to a human being only in this case it is the operator of a machine evaluating sensor data and adjusting mechanics.
• Mechanics — are the parts of the machine that direct it’s behavior. They respond to signals issued by the code/operator. Most importantly on an ascending airplane it’s flaps are repositioned so the tail is raised up and the nose is lowered.

All three of these elements are present in all automated machinery and the successful operation of the machine is completely dependent on each of them performing error-free. Problems, sometimes lethal ones, arise whenever errors occur in any one of the three components or as they exchange messages among each other.

For example, Sensors can incorrectly measure the environment. Code can incorrectly process the data it receives from the sensors and the Mechanics can respond incorrectly to the signals it receives from the code.

In the case of catastrophic accidents problems are often systemic when all three components behave in a way not intended by the manufacturer. Even rigorous testing is not a fail-safe mechanism to avoid mistakes from occuring in complex systems. There’s simply no way of anticipating “all” the events that might occur in the real environment and cause a system to fail. Both automated airplanes and self-driving cars behave as described above and both can suffer from the same malady of complexity.

The code is particularly problematic because it can be written with more or less complexity and complexity breeds errors. First of all, because the code is of a recent vintage it’s most likely object-oriented (OOP) rather than the older procedural style code. That means the details of the code, that actually process the input data from the sensors and sends signals (i.e. messages) to the mechanics of the airplane, relies on object orient design principles, the most important being; Abstraction, Encapsulation and Inheritance:

• Abstraction means working with something we know how to use without knowing how it works internally. A good example is a television set. We don’t need to know the inner workings of a TV, in order to use it. MCAS programmers didn’t need to know the code in objects that processed data collected by sensors. All they needed to know is that it sent “appropriate” signals to the tail mechanism objects based on the characteristics of the data collected by sensors.
• Encapsulation is one of the main concepts in OOP. It is also called “information hiding”. An object has to provide its users only with the essential information for manipulation, without the internal details. A Secretary using a Laptop only knows about its screen, keyboard and mouse. Everything else is hidden internally under the cover. She does not know about the inner workings of Laptop, because she doesn’t need to, and if she does, she might make a mess. Therefore parts of the properties and methods of the laptop remain hidden to her.
• Inheritance is a fundamental principle of object-oriented programming. It allows a class of code to “inherit” (behavior or characteristics) of another, more general class. For example, a lion belongs to the biological family of cats (Felidae). All cats that have four paws, are predators and hunt their prey. This functionality can be coded once in the Felidae class and all its predators can reuse it — Tiger, Puma, Bobcat, etc. Inheritance is described as is-kind-of relationship, e.g. Tiger is kind of Animal.

The developers of the Boeing MCAS code may have never looked at the actual code they used to process data received from sensors. All they may have done was look at the input and output specifications of objects used in earlier applications and chose to “reuse” those objects. They then made sure that when they tested the reused code with data from actual 737 MAX 8 sensors, the code directed the mechanics, e.g. tail flaps, of a 737 MAX 8 as they expected. This meant they needed some type of calibration that would adjusted the tail flaps so the nose of the plan was adjusted in a way that changed the plane’s AOA appropriately. All that meant the software needed accurate measurements of the position of the tail flaps and nose, in addition to a measurement of an appropriate position to adjust the nose, to begin the calculations. Much of that is achieved by the experience and feel of human pilots.

These particular crashes are a real problem, however, not because of the complexity of the technology that caused them (i.e. the MCAS ). They’re a problem because of the economics that likely drove the development of technology. Those economics are as old as ones used to operate Adam Smith’s 1776 pin factory which basically advocates substitution of technology for labor to lower unit labor costs. Lower unit labor costs mean less labor, like fewer pilots observing behavior of the airplane. More problematic, those economics are so ingrained in the thought processes of managers of U.S. businesses they are sure to create problems with the production of products with similar requirements to the 737 MAX 8, like self-driving cars.

Imagine a self driving car accidentally approaching a steel meridian separating lanes of traffic on a busy highway. The car will be equipped with it’s own MCAS-like technology to process data received from it’s sensors and send the results to mechanical systems to direct the car away from the meridian. Just like the code used in the 737 MAX 8 MCAS, the code in the self driving car will have been developed using object oriented principles to maximize the economics of it’s development but just like the process of the MAX 8 sensor data causing the nose of the plane to decend directly into the ground the self-driving car could be violently turned into adjoining lanes of traffic and even completely off the highway.

For sure Cadillac, Chevy, Buick and other GM brand design teams will be testing the code of their self driving cars to ensure they don’t make “violent” adjustments if redirected by their MCAS. But Tesla likely tested the code of it’s Model S electric car before the car’s cameras (i.e. sensors) failed to distinguish the white side of a turning tractor-trailer from a brightly lit sky ( i.e. code) and didn’t automatically activate its brakes ( i.e. message it’s mechanics). Joshua D. Brown, of Canton, Ohio, died in the accident May 7 in Williston, Florida. The accident of one driver didn’t get the same publicity as 230+ passengers dying on two commercial air flights but the cause was not dissimilar.

Cadillac executives may be right, self driving cars may be the way of the future but if they are it will be because of the economics of their design and development, not their passenger safety.