CPS Group researchers are working with industry partners to investigate ways to integrate the mental states of air pilots with machine learning and decision-making. The Cognitive Human-Machine Interface and Interaction (CHMI2) system analyses data inputs to determine the pilot’s state of mind and decide how much computer support is required to fly the plane. A pilot’s eye-movement, facial expression, brain activity, body temperature, voice patterns, posture and exertion as well as emotional stress indicators like those used in lie-detector tests are among the kinds of information that are assessed by the system.
Professor Roberto Sabatini, the research leader of the CPS Group at RMIT, said the computer system is used to help with flight control and collision avoidance. “If computers are helping us fly planes they need to consider more than just weather conditions and other aircraft movements, they also need to understand the condition of the pilot they’re working alongside and change the balance of human-computer control accordingly,” Sabatini said. “In the most obvious example, the system can tell if a pilot is severely incapacitated or unconscious then take full control of the aircraft, but there are many more subtle instances of where they can work together and maximise the combined human-machine team performance.”
The system can also be used to support air traffic controllers and may have applications for single person control of multiple unmanned aircraft. Featuring algorithms and neural networks modelled on the human brain, which allows it to learn and improve itself with repeated interactions, Sabatini said the computer develops a partnership with the user. “The emerging concept of human-machine teaming has massive potential to extend our abilities and performance in aerospace, defence, healthcare and a whole range of industries. This is a significant step forward in enabling true, two-way collaboration between humans and machines,” he said.
The research is being conducted with industry partners Northrop Grumman Corporation, Thales Australia and the Australian Defence Science and Technology (DST) group at RMIT’s Aerospace Intelligent and Autonomous Systems Laboratory in the School of Engineering. The research team’s paper on “Cognitive Human-Machine Interfaces and Interactions for Avionics and Air Traffic Management Systems” was awarded the prize for best paper at latest IEEE International Workshop on Metrology for Aerospace.
Prof. Sabatini said that the technology was marrying human creativity and flexibility with the tactical precision and information acquisition and synthesising skills of machines. “Both need to work together to achieve optimum performance and that’s where we’re taking this exciting technology – towards a whole range of real-life applications.
