Hybrid visual information analysis for on-site occupational hazards identification: A case study on stairway safety

Slip, trip and fall (STF) are the leading type of fatalities in the construction industry and most occupational STF accidents on stairs occur when construction workers unconsciously violate safety rules due to inattentiveness and hastiness. Thus, computer-aided monitoring systems is becoming increasingly important for on-site occupational safety management. However, construction site scenes generally contain a variety of different entities (e.g., individuals, facilities), which places a higher demand on the hybrid visual information understanding capability of the scenes of computer-aided monitoring systems. This paper presents a novel hybrid visual information analysis framework. First, a visual information extraction module integrating the state-of-the-art instance segmentation and pose estimation models is proposed to obtain hybrid on-site entities information. Subsequently, hazards are identified with an original geometric relationship analysis algorithm and the identification performance is further enhanced using time series analysis. Two hybrid visual information analysis frameworks, i.e., HVIA-BU and HVIA-TD, are proposed based on bottom-up and top-down pose estimation models, respectively. We implemented and experimentally evaluated different architectures of each framework in terms of both identification performance and inference speed to address the different on-site hardware requirements. As an initial application of the proposed framework for on-site occupational hazards identification, we performed the experiments with handrail-related compliance as a case study. The proposed hybrid visual information analysis framework HVIA-TD achieved high precision (0.9826) and recall (0.9535), outperforming the single visual information analysis framework SVIA (with a precision of 0.9551 and a recall of 0.9121).