A Framework for Human-Robot Teaming Performance Prediction: Reinforcement Learning and Eye Movement Analysis

Abstract In modern manufacturing environments, mass customization demands a balance between automation and human involvement. Human performance is influenced by the task load, with high loads leading to stress and low loads leading to disengagement, both of which can negatively affect performance. Predicting human engagement and performance can allow for adjustments in the team’s actions, leading to optimal performance. This paper proposes a framework for predicting the performance of a human-robot team in a quality control task based on task load, engagement, and eye movement data. The proposed framework employs a computational pipeline and eye movement data to monitor task load. It uses the NASA TLX questionnaire to assess participants’ task load and reinforcement learning to derive task-specific weights based on their performance. Then, the eye movement data is used to classify performance. The framework is evaluated with data collected from 16 participants performing a quality control task with a collaborative robot, in two scenarios. The study found that the framework predicts human-robot team performance with an accuracy of 96.88%. It also explores the potential of replacing the physiological data with a wristband with eye gaze for performance prediction due to challenges to record the eye movement data.