Simulator based performance metrics to estimate reliability of control room operators

Chemical processes rely on several layers of protection to prevent accidents. One of the most important layers of protection is human operators. Human errors are a key contributor in a majority of accidents today. Estimation of human failure probabilities is a challenge due to the numerous drivers of human error, and still heavily dependent on expert judgment. In this paper, we propose a strategy to estimate the reliability of control room operators by measuring their control performance on a process simulator. The performance of the operator is translated to two metrics – margin-of-failure and available-time to respond to process events – which can be calculated using process operations data that can be generated from training simulator based studies. These metrics offer a qualitative estimate of operators’ reliability. We conducted a set of experiments involving 128 students of differing capabilities from two different institutions and tasked to control a simulated ethanol production plant. Our results demonstrate that differences in the performance of expert vs. novice student operators can be clearly distinguished using the metrics.