Developing a method for evaluating crew resource management skills: A European perspective

The European Commission in conjunction with the European Joint Aviation Authorities (Human Factors Project Advisory Group) has been sponsoring a series of studies investigating a culturally robust method for the evaluation of pilots' nontechnical skills (NTS) for multicrew operations. This article outlines the development of a European NTS behavioral marker system for crew resource management evaluation called NOTECHS and presents preliminary results from an ongoing test phase of this system (Joint Aviation Requirements Translation and Elaboration of Legislation [JARTEL]). The JARTEL project has involved 105 instructors from 14 European airlines who were given a short training session to use the NOTECHS system. After the training phase, these instructor pilots used the system to evaluate the individual CRM skills of captains and first officers in 8 different video scenarios filmed in a Boeing 757 simulator. Issues relating to rater training, reliability, and accuracy, as well as the instructors' opinions on the acceptability of the method, are discussed.