As mentioned before, the mayor challenge of future AI systems in the cockpit is tackling these edge cases. These edge cases can be especially difficult to identify and understand due to the fact that they tend to be situations never before experienced. This can be a great challenge as AI models learn from data and are only as good as the data they get.
A good example of what could happen when working with bad data is the Lion Air 737 Max that crashed in October 2018. Although the investigation is still ongoing, one of the preliminary reasons for the crash that it is being investigated is faulty data from the sensor. Authorities believe that one of the angle of attack sensors was feeding the MCAS faulty data, making it think the aircraft was going to stall. The MCAS then proceed to dive the plane in order to gain speed and avoid the stall. Lack of training and familiarity of MCAS among the crew also contributed to the crash.
As another example, in 2009, Air France flight 447, an Airbus A330, stalled and crashed in the Atlantic Ocean. The crash occurred after an inconsistency between the airspeed measurements (IAS) due to ice crystals obstructing the aircraft’s pitot tubes, a previously unseen situation. This inconsistency made the autopilot disengage and switch to an alternate mode also unfamiliar to the crew. Unfortunately, they incorrectly assessed the situation and focused their attention entirely on airspeed inconsistency. Without following the correct procedures for IAS inconsistency, they decided to rise the nose of the aircraft. The A330, similar to the 737 Max, is equipped with a non-stall system, but the crew was unaware (Mode Confusion) that said system disconnected due to the computer having switched to an alternate mode. Ultimately, the aircraft did enter into a stall due to the steep climb and did not recover.
From these two recent crashes, we can extract two main challenges for AI autonomous aircrafts, with the first being technical. Both accidents mentioned started off with incorrect data provided by different sensors. Again, AI is only as good as its input data. Sensors fail and can be defective, so we should assume some bad input data as a principle. Therefore, the AI needs to detect both when a sensor can malfunction and when this could be the sign of some major problem. This is one of the biggest, current obstacles of AI. Humans are still needed in edge case situations because, as humans, we have the distinct ability to create accurate cause-effect models of how something works. We have common sense knowledge that can be applied in making reasoned logical inferences that, in many cases, leads to the solution of a previously unseen problem.
The second challenge for AI autonomous aircrafts is understanding and reviewing how automation affects pilot abilities. Until we achieve fully autonomous aircraft, humans and AI are going to share the cockpit. As seen in recent accidents, pilot abilities have been posited to have degraded dangerously low due to automation. Although they still go through rigorous training, pilots nowadays rarely take manual control of the aircraft during a flight – only a couple of minutes during takeoff and landing. This problem is worsened by, as many studies suggest, the fact that, as pilots are not involved directly with the operation (more monitoring role), they can lose track of what is happening and what the aircraft is doing. This may influence a pilot’s ability to react when a problem rises. The accident of Air France 447 is a great example of these challenges: the captain had logged in the past 6 months 346 hours from which only about 4 hours was he actually in control of the aircraft; when the autopilot disengaged, the crew had problems identifying its cause and were unaware that the alternate mode meant full control of the aircraft, meaning they could induce a stall by rising the nose of the aircraft too much. No one argues that automation benefits aviation safety, but many organisations have stated that changes need to be made to crew training. Many have emphasised the need of crews to maintain the ability to fly the aircraft manually in any phase of the flight in addition to better understanding of the systems the aircraft is equipped with, how they work and what can they do.