Utilizing 360-Degree Images for Synthetic Data Generation in Construction Scenarios Aqsa Sabir Chung Ang University, Korea, Republic of Rahat Hussain Chung Ang University, Korea, Republic of Syed Farhan Alam Zaidi Chung Ang University, Korea, Republic of Akeem Pedro Chung Ang University, Korea, Republic of Mehrtash Soltani Chung Ang University, Korea, Republic of Dongmin Lee Chung Ang University, Korea, Republic of Chansik Park Chung Ang University, Korea, Republic of This is a section of CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality (DOI: 10.36253/979-12-215-0289-3) by Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi Firenze University Press Florence 2023 https://doi.org/10.36253/10.36253/979-12-215-0289-3.70

Available for academic research purposes

Open Access

Copyright Author(s)

Content licence CC BY-NC 4.0

Metadata licence CC0 1.0

This is original content, published for academic research purposes

 Digital edition XML powered by Booksflow

Computer vision-based safety monitoring requires machine learning models trained on generalized datasets covering various viewpoints, surface properties, and lighting conditions. However, capturing high-quality and extensive datasets for some construction scenarios is challenging at real job sites due to the risky nature of construction scenarios. Previous methods have proposed synthetic data generation techniques involving 2D background randomization with virtual objects in game-based engines. While there has been extensive work on utilizing 360-degree images for various purposes, no study has yet employed 360-degree images for generating synthetic data specifically tailored for construction sites. To improve the synthetic data generation process, this study proposes a 360-degree images-based synthetic data generation approach using Unity 3D game engine. The approach efficiently generates a sizable dataset with better dimensions and scaling, encompassing a range of camera positions with randomized lighting intensities. To check the effectiveness of our proposed method, we conducted a subjective evaluation, considering three key factors: object positioning, scaling in terms of object respective size, and the overall size of the generated dataset. The synthesized images illustrate the visual improvement in all three factors. By offering an improved data generation method for training safety-focused computer vision models, this research has the potential to significantly enhance the automation of the construction safety monitoring process, and hence, this method can bring substantial benefits to the construction industry by improving operational efficiency and reinforcing safety measures for workers

 360-Degree Images Computer Vision Synthetic Data Generation Game Engine Object Detection Construction Safety Monitoring

It is available online at https://doi.org/10.36253/10.36253/979-12-215-0289-3.70

References Assadzadeh, A., Arashpour, M., Brilakis, I., Ngo, T., & Konstantinou, E. (2022). Vision-based excavator pose estimation using synthetically generated datasets with domain randomization. Automation in Construction, 134, 104089. 10.1016/j.autcon.2021.104089 Barrera-Animas, A. Y., & Davila Delgado, J. M. (2023). Generating real-world-like labelled synthetic datasets for construction site applications. Automation in Construction, 151, 104850. 10.1016/j.autcon.2023.104850 Borkman, S., Crespi, A., Dhakad, S., Ganguly, S., Hogins, J., Jhang, Y.-C., Kamalzadeh, M., Li, B., Leal, S., Parisi, P., Romero, C., Smith, W., Thaman, A., Warren, S., & Yadav, N. (2021). Unity Perception: Generate Synthetic Data for Computer Vision. 10.48550/arXiv.2107.04259 Choi, W., & Pyun, S. (2021). Synthetic Training Data Generation for Fault Detection Based on Deep Learning. Geophysics and Geophysical Exploration, 24(3), 89–97. 10.7582/GGE.2021.24.3.089 Frolov, V., Faizov, B., Shakhuro, V., Sanzharov, V., Konushin, A., Galaktionov, V., & Voloboy, A. (2022). Image Synthesis Pipeline for CNN-Based Sensing Systems. Sensors, 22(6), 2080. 10.3390/s22062080 Kim, A., Lee, K., Lee, S., Song, J., Kwon, S., & Chung, S. (2022). Synthetic Data and Computer-Vision-Based Automated Quality Inspection System for Reused Scaffolding. Applied Sciences, 12(19), 10097. 10.3390/app121910097 Kim, J., Kim, D., Shah, J., & Lee, S. (2022). Training a Visual Scene Understanding Model Only with Synthetic Construction Images. Computing in Civil Engineering 2021, 221–229. 10.1061/9780784483893.028 Lee, J. G., Asce, S. M., Hwang, J., Chi, S., Asce, M., & Seo, J. (2022). Synthetic Image Dataset Development for Vision-Based Construction Equipment Detection. Computing in Civil Engineering, 36(5). 10.1061/(ASCE)CP.1943-5487.0001035 Li, Y., Wei, H., Han, Z., Jiang, N., Wang, W., & Huang, J. (2022). Computer Vision-Based Hazard Identification of Construction Site Using Visual Relationship Detection and Ontology. Buildings, 12(6), 857. Neuhausen, M., Herbers, P., & König, M. (2020). Using synthetic data to improve and evaluate the tracking performance of construction workers on site. Applied Sciences (Switzerland), 10(14), 4948. 10.3390/app10144948 Oh, X., Loh, L., Foong, S., Koh, Z. B. A., Ng, K. L., Tan, P. K., Toh, P. L. P., & Tan, U. X. (2021). Initialisation of Autonomous Aircraft Visual Inspection Systems via CNN-Based Camera Pose Estimation. Proceedings - IEEE International Conference on Robotics and Automation, 2021-May, 11047–11053. 10.1109/ICRA48506.2021.9561575 Rho, J., Park, M., & Lee, H.-S. (2020). Automated construction progress management using computer vision-based CNN model and BIM. Korean Journal of Construction Engineering and Management, 21(5), 11–19. Sami Ur Rehman, M., Shafiq, M. T., & Ullah, F. (2022). Automated Computer Vision-Based Construction Progress Monitoring: A Systematic Review. Buildings, 12(7), 1037. Siu, C., Wang, M., & Cheng, J. C. P. (2022). A framework for synthetic image generation and augmentation for improving automatic sewer pipe defect detection. Automation in Construction, 137, 104213. 10.1016/j.autcon.2022.104213 Soltani, M. M., Zhu, Z., & Hammad, A. (2016). Automated annotation for visual recognition of construction resources using synthetic images. Automation in Construction, 62, 14–23. 10.1016/j.autcon.2015.10.002 Sutjaritvorakul, T., Vierling, A., & Berns, K. (2020). Data-driven worker detection from load-view crane camera. Proceedings of the 37th International Symposium on Automation and Robotics in Construction, ISARC 2020: From Demonstration to Practical Use - To New Stage of Construction Robot, 864–871. 10.22260/isarc2020/0119 Tohidifar, A., Fatemeh Saffari, S., & Kim, D. (2022). Fake it till you make it: Training Deep Neural Networks for Worker Detection using Synthetic Data. The 29th EG-ICE International Workshop on Intelligent Computing in Engineering, 386–396. 10.7146/AUL.455.C229 Wei, Y., & Akinci, B. (2022). Synthetic Image Data Generation for Semantic Understanding in Everchanging Scenes Using BIM and Unreal Engine. Computing in Civil Engineering 2021, 934–941. 10.1061/9780784483893.115 Wong, M. Z., Kunii, K., Baylis, M., Ong, W. H., Kroupa, P., & Koller, S. (2019). Synthetic dataset generation for object-to-model deep learning in industrial applications. PeerJ Computer Science, 2019(10), e222. 10.7717/peerj-cs.222 Zhang, J., Fukuda, T., & Yabuki, N. (2022). Automatic generation of synthetic datasets from a city digital twin for use in the instance segmentation of building facades. Journal of Computational Design and Engineering, 9(5), 1737–1755. 10.1093/jcde/qwac086