Predictive Safety Monitoring for Lifting Operations with Vision-Based Crane-Worker Conflict Prediction Peter Kok-Yiu Wong The Hong Kong University of Science and Technology, Hong Kong Chin Pok Lam University of Hong KongThe Hong Kong University of Science and Technology, Hong Kong Yin Ni Lee University of Hong KongThe Hong Kong University of Science and Technology, Hong Kong Chung Lam Ting University of Hong KongThe Hong Kong University of Science and Technology, Hong Kong Jack C. P. Cheng University of Hong KongThe Hong Kong University of Science and Technology, Hong Kong Pak Him Leung AutoSafe Limited, Hong Kong This is a section of CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality (DOI: 10.36253/979-12-215-0289-3) by Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi Firenze University Press Florence 2023 https://doi.org/10.36253/10.36253/979-12-215-0289-3.64

Available for academic research purposes

Open Access

Copyright Author(s)

Content licence CC BY-NC 4.0

Metadata licence CC0 1.0

This is original content, published for academic research purposes

 Digital edition XML powered by Booksflow

Construction industry has reported among the highest accident and fatality rates over the past decade. In particular, crane lifting is a notably hazardous operation on construction sites, causing fatal accidents like workers being struck by the boom or objects fallen from tower cranes. Manual monitoring by on-site safety officers is labour-intensive and error-prone, while incorporating computer vision techniques into surveillance cameras would enable more automatic and continuous monitoring of construction site operations. However, existing studies for lifting safety mainly detect the presence of individual objects (e.g. workers, crane components), while a methodology is needed to predict their potential collision more proactively before accidents happen. This paper develops a vision-based framework for predictive lifting safety monitoring, including three modules: (1) object detection and classification: targeting at hook and lifting materials to enable danger zone estimation, along with workers and their personal protective equipment; (2) worker movement tracking and prediction: analyzing the historical moving trajectory of each unique worker to foresee his/her future movement in certain period ahead; (3) multi-level safety assessment: issuing predictive warning in real-time upon any crane-worker conflict foreseen. The proposed framework is applicable to real-time site video processing and enables end-to-end lifting safety monitoring with instant alerting upon unsafe scenarios observed. Importantly, the proposed framework predicts the future movement of workers to proactively identify potential site hazard, in order to trigger earlier safety alert for more timely decision-making. With a large video dataset capturing tower crane operations, the proposed framework demonstrates competitive accuracy and computational efficiency in crane-worker conflict prediction, validating its practicality for real-time lifting safety monitoring

 Computer Vision; Construction Safety Monitoring; Crane-Worker Conflict Prediction; Deep Learning; Predictive Safety Assessment; Trajectory Tracking

It is available online at https://doi.org/10.36253/10.36253/979-12-215-0289-3.64

References Brunetti, A., Buongiorno, D., Trotta, G. F., & Bevilacqua, V. (2018). Computer vision and deep learning techniques for pedestrian detection and tracking: A survey. Neurocomputing, 300, 17-33. 10.1016/j.neucom.2018.01.092 Cheng, J. P., Wong, P. K. Y., Luo, H., Wang, M., & Leung, P. H. (2022). Vision-based monitoring of site safety compliance based on worker re-identification and personal protective equipment classification. Automation in Construction, 139, 104312. 10.1016/j.autcon.2022.104312 Chian, E. Y. T., Goh, Y. M., Tian, J., & Guo, B. H. (2022). Dynamic identification of crane load fall zone: A computer vision approach. Safety science, 156, 105904. 10.1016/j.ssci.2022.105904 Fang, Q., Li, H., Luo, X., Ding, L., Luo, H., Rose, T. M., & An, W. (2018). Detecting non-hardhat-use by a deep learning method from far-field surveillance videos. Automation in construction, 85, 1-9. 10.1016/j.autcon.2017.09.018 Golovina, O., Teizer, J., & Pradhananga, N. (2016). Heat map generation for predictive safety planning: Preventing struck-by and near miss interactions between workers-on-foot and construction equipment. Automation in construction, 71, 99-115. 10.1016/j.autcon.2016.03.008 HKSARG Development Bureau (2020). Lifting Safety Handbook. Construction Safety Week 2020. [Online]. Available: https://www.cic.hk/files/page/51/Lifting%20Safety%20Handbook%20%E5%90%8A%E9%81%8B%E5%AE%89%E5%85%A8%E6%89%8B%E5%86%8A.pdf. [Accessed: Oct. 29, 2022] HKSARG Labour Department (2018). Occupational Safety and Health Statistics: Bulletin Issue No. 18. [Online]. Available: https://www.labour.gov.hk/eng/osh/pdf/Bulletin2017_issue18_eng.pdf. [Accessed: 18-Dec-2022] Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer I., Wojna Z., Song Y., Guadarrama S., & Murphy, K. (2017). Speed/accuracy trade-offs for modern convolutional object detectors. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7310-7311). 10.1109/cvpr.2017.351 Jeelani, I., Asadi, K., Ramshankar, H., Han, K., & Albert, A. (2021). Real-time vision-based worker localization & hazard detection for construction. Automation in Construction, 121, 103448. 10.1016/j.autcon.2020.103448 Kim D., Liu M., Lee S. H., & Kamat V. R. (2019). Remote proximity monitoring between mobile construction resources using camera-mounted uavs. Automation in Construction, vol. 99, pp. 168–182. 10.1016/j.autcon.2018.12.014 Liu, W., Meng, Q., Li, Z., & Hu, X. (2021). Applications of computer vision in monitoring the unsafe behavior of construction workers: Current status and challenges. Buildings, 11(9), 409. 10.3390/buildings11090409 Luo, H., Wang, M., Wong, P. K. Y., Tang, J., & Cheng, J. C. (2021). Construction machine pose prediction considering historical motions and activity attributes using gated recurrent unit (GRU). Automation in Construction, 121, 103444. 10.1016/j.autcon.2020.103444 Luo, X., Li, H., Wang, H., Wu, Z., Dai, F., & Cao, D. (2019). Vision-based detection and visualization of dynamic workspaces. Automation in Construction, 104, 1-13. 10.1016/j.autcon.2019.04.001 Memarzadeh, M., Golparvar-Fard, M., & Niebles, J. C. (2013). Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors. Automation in Construction, 32, 24-37. 10.1016/j.autcon.2012.12.002 Nath, N. D., Behzadan, A. H., & Paal, S. G. (2020). Deep learning for site safety: Real-time detection of personal protective equipment. Automation in Construction, 112, 103085. 10.1016/j.autcon.2020.103085 Shafique, M., & Rafiq, M. (2019). An overview of construction occupational accidents in Hong Kong: A recent trend and future perspectives. Applied Sciences, 9(10), 2069. 10.3390/app9102069 Son, H., Seong, H., Choi, H., & Kim, C. (2019). Real-time vision-based warning system for prevention of collisions between workers and heavy equipment. Journal of Computing in Civil Engineering, 33(5), 04019029. 10.1061/(ASCE)CP.1943-5487.0000845 U.S. Bureau of Labor Statistics (BLS) (2018). National Census of Fatal Occupational Injuries in 2017. [Online]. Available: www.bls.gov/iif/oshcfoil.htm. [Accessed: 18-Dec-2022] Wojke, N., Bewley, A., & Paulus, D. (2017, September). Simple online and realtime tracking with a deep association metric. In 2017 IEEE international conference on image processing (ICIP) (pp. 3645-3649). IEEE. 10.1109/icip.2017.8296962 Wong, P. K. Y., Luo, H., Wang, M., Leung, P. H., & Cheng, J. C. (2021). Recognition of pedestrian trajectories and attributes with computer vision and deep learning techniques. Advanced Engineering Informatics, 49, 101356. 10.1016/j.aei.2021.101356 Zhang, M., Cao, Z., Yang, Z., & Zhao, X. (2020). Utilizing computer vision and fuzzy inference to evaluate level of collision safety for workers and equipment in a dynamic environment. Journal of Construction Engineering and Management, 146(6), 04020051. 10.1061/(ASCE)CO.1943-7862.0001802