How Facial Thermography Can Estimate A Pilot’s Mental Workload

In industries where excessive mental workload can have tragic consequences, researchers at the University of Nottingham have been investigating non-invasive and non-intrusive methods to assess cognitive demand. Their new research suggests that easily collected physiological measurements, including facial thermography and pupil diameter, correlate strongly with the operator’s perception of task demand and difficulty.

With the skies becoming ever more congested, pilots and air traffic controllers have to juggle a number of tasks, with multiple tasks demanding both skill and situational awareness. Typically, operators would be asked to assess the difficulty of a task by responding to a questionnaire once the task is complete, or during a lull in the activity. Researchers at the University of Nottingham’s Institute for Aerospace Technology and Faculty of Engineering were interested in developing a method that can estimate the demands being made without having to interrupt a task or distract the operator. Of all the methods used, it was found that facial thermography and pupil diameter offered the most promise, as they are the least intrusive. When used in conjunction with machine learning, they offer a better correlation to reported workloads.

When a volunteer was asked to take part in a computer-based task combining cognitive demand with spatial awareness, the thermal camera clearly showed that as the task became more difficult there was a noticeable reduction in the temperature over the sinuses, and particularly at the tip of the nose. Simultaneous eye-tracking showed that pupils dilated as the demand increased. The drop in temperature could be due to a diversion of blood from the face to the brain or may be due to changes in convective cooling of the nose due to variations in breathing rate. To reduce the occurrence of false positives, the two measures were combined and found to correlate strongly with reported demand.

The team wanted to take the research further to develop automated systems to assist operators and their supervisors at times of excessive workload.

According to Adrian Marinescu, who conducted the study as part of his Ph.D., Machine Learning is key to the accuracy of the technique. “We know that there is a correlation between workload and nose temperature, breathing rate, and pupil diameter, but people are highly variable. When we used a machine learning algorithm, which takes the individual’s responses to stimuli into account, the assessment of workload was much more accurate. The next stage would be to develop an algorithm to assist the operator in real time.”

Dr. Alastair Campbell Ritchie, one of the supervisors of the study, commented: “It’s very interesting to see the result, particularly because the technology used is becoming more accurate and more accessible. It shows that Bioengineers and Human Factors specialists can work together to make transport and workplaces safer.”

Professor Sarah Sharples, Professor of Human Factors, and supervisor and initiator of the study said, “The measurement of workload without needing to interrupt people to ask them to report how busy they are, has been challenging human factors specialists for many years. By bringing together our expertise in bioengineering, human factors and machine learning, we have developed a much better understanding of how physical changes associated with workloads manifest themselves as physiological symptoms, and how these symptoms translate into the parameters that we can measure.”

These findings are described in the article entitled Physiological Parameter Response to Variation of Mental Workload, published in the journal Human Factors. This work was led by Adrian Cornelius Marinescu and Alastair Campbell Ritchie from the University of Nottingham.