The establishment of smart construction sites employing smart technologies for real-time interconnection, communication, and interaction has emerged as an interesting topic for researchers and practitioners in the architecture, engineering, and construction industry as they contribute to effective, efficient, and high-quality on-site management. However, little consideration has been involved in the comprehensive management of health, safety, and environment (HSE) issues using smart technologies. To address this gap, this study first performs a systematic procedure to identify the articles (results = 325) associated with the on-site application of smart technologies for HSE and conducts a quantitative and qualitative analysis on their research trend and interests. A bibliometric analysis of publication numbers by year, country and region, and journal presents the research state quo and trends, and a keyword co-occurrence analysis is conducted to show the interests of this topic. Based on these interests, different smart technologies including techniques and devices are categorized for on-site HSE management objectives. Moreover, a comprehensive framework for smart technologies in HSE management and a blueprint for HSE-oriented smart construction site establishment are put forward. This study develops a basis to understand the current application of smart technologies in HSE management and contributes to the implementation of smart construction sites for sustainable construction.