Human-in-the-Loop Digital Twin Framework for Assessing Ergonomic Implications of Exoskeletons Abiola Akanmu Virginia Tech/Myers Lawson School of Construction, United States Adedeji Afolabi Virginia Tech/Myers Lawson School of Construction, United States Akinwale Okunola Virginia Tech/Myers Lawson School of Construction, United States This is a section of CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality (DOI: 10.36253/979-12-215-0289-3) by Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi Firenze University Press Florence 2023 https://doi.org/10.36253/10.36253/979-12-215-0289-3.121

Available for academic research purposes

Open Access

Copyright Author(s)

Content licence CC BY-NC 4.0

Metadata licence CC0 1.0

This is original content, published for academic research purposes

 Digital edition XML powered by Booksflow

Exoskeletons are increasingly being recognized as ergonomic solutions for work-related musculoskeletal disorders in the construction industry. However, users of active back-support exoskeletons are susceptible to various physical and psychological risks, which could be exoskeleton type-or task-dependent. A test bed is needed to enable deployment and assessment of risks associated with exoskeleton-use for construction tasks. This study aims to develop a human-in-the-loop digital twin framework for assessing ergonomic risks associated with the use of active back-support exoskeletons for construction work. A literature review was conducted to identify risks associated with exoskeletons and the technologies for quantifying the risks. This informed the development of a system architecture describing the enabling technologies and their roles in assessing risks associated with active back-support exoskeletons. Semi-structured interviews were conducted to identify construction tasks that are most suitable for active back-support exoskeletons. Based on the identified tasks, a laboratory experiment was conducted to quantify the risks associated with the use of a commercially available active back-support exoskeleton for carpentry framing tasks. The efficacy of the digital twin framework is demonstrated with an example of the classification of exertion levels due to exoskeleton-use using a 1D-convolutional neural network. The study showcases the potential of digital twins for comprehensive ergonomic assessment, enabling stakeholders to proactively address ergonomic risks and optimize the use of exoskeletons in the construction industry. The framework demonstrates the significance of evidence-based decision-making in enhancing workforce health and safety

 Digital twin ergonomics exertion exoskeletons risk assessment sensing technologies work-related musculoskeletal disorders

It is available online at https://doi.org/10.36253/10.36253/979-12-215-0289-3.121

References Adu, P. (2019). A step-by-step guide to qualitative data coding. Routledge. Akanmu, A. A., Olayiwola, J., Ogunseiju, O., & McFeeters, D. (2020). Cyber-physical postural training system for construction workers. Automation in Construction, 117. 10.1016/j.autcon.2020.103272 Alabdulkarim, S., Kim, S., & Nussbaum, M. A. (2019). Effects of exoskeleton design and precision requirements on physical demands and quality in a simulated overhead drilling task. Applied Ergonomics, 80, 136-145. 10.1016/j.apergo.2019.05.014 Albert, J. A., Herdick, A., Brahms, C. M., Granacher, U., & Arnrich, B. (2021). Using Machine Learning to Predict Perceived Exertion During Resistance Training With Wearable Heart Rate and Movement Sensors. 2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). 10.1109/BIBM52615.2021.9669577. Alemi, M. M., Madinei, S., Kim, S., Srinivasan, D., & Nussbaum, M. A. (2020). Effects of Two Passive Back-Support Exoskeletons on Muscle Activity, Energy Expenditure, and Subjective Assessments During Repetitive Lifting. Hum Factors, 62(3), 458-474. 10.1177/0018720819897669 Alzahab, N. A., Apollonio, L., Di Iorio, A., Alshalak, M., Iarlori, S., Ferracuti, F., Monteriu, A., & Porcaro, C. (2021). Hybrid Deep Learning (hDL)-Based Brain-Computer Interface (BCI) Systems: A Systematic Review. Brain Sciences, 11(1), 75. ARTN 7510.3390/brainsci11010075 Antwi-Afari, M. F., Anwer, S., Umer, W., Mi, H. Y., Yu, Y. T., Moon, S., & Hossain, U. (2023). Machine learning-based identification and classification of physical fatigue levels: A novel method based on a wearable insole device. International Journal of Industrial Ergonomics, 93. 10.1016/j.ergon.2022.103404 Baltrusch, S. J., Houdijk, H., van Dieen, J. H., & de Kruif, J. T. C. M. (2021). Passive Trunk Exoskeleton Acceptability and Effects on Self-efficacy in Employees with Low-Back Pain: A Mixed Method Approach. Journal of Occupational Rehabilitation, 31(1), 129-141. 10.1007/s10926-020-09891-1 Bangaru, S. S., Wang, C., Busam, S. A., & Aghazadeh, F. (2021). ANN-based automated scaffold builder activity recognition through wearable EMG and IMU sensors. Automation in Construction, 126, 103653. 10.1016/j.autcon.2021.103653. BLS. (2023). IIF Databases : U.S. Bureau of Labor Statistics. https://www.bls.gov/iif/data.htm. Borg, G. A. (1982). Psychophysical bases of perceived exertion. Medicine and science in sports and exercise, 14(5), 377-381. 10.1249/00005768-198205000-00012 Chowdhury, A. K., Tjondronegoro, D., Chandran, V., Zhang, J., & Trost, S. G. (2019). Prediction of relative physical activity intensity using multimodal sensing of physiological data. Sensors, 19(20), 4509. 10.3390/s19204509. Cumplido-Trasmonte, C., Barquin-Santos, E., Garces-Castellote, E., Gor-Garcia-Fogeda, M. D., Plaza-Flores, A., Hernandez-Melero, M., Gutierrez-Ayala, A., Cano-de-la-Cuerda, R., Lopez-Moron, A. L., & Garcia-Armada, E. (2023). Safety and usability of the MAK exoskeleton in patients with stroke. Physiotherapy Research International. 10.1002/pri.2038 de Looze, M. P., Bosch, T., Krause, F., Stadler, K. S., & O'Sullivan, L. W. (2016). Exoskeletons for industrial application and their potential effects on physical work load. Ergonomics, 59(5), 671-681. 10.1080/00140139.2015.1081988 Delgado, A. D., Escalon, M. X., Bryce, T. N., Weinrauch, W., Suarez, S. J., & Kozlowski, A. J. (2020). Safety and feasibility of exoskeleton-assisted walking during acute/sub-acute SCI in an inpatient rehabilitation facility: A single-group preliminary study. J Spinal Cord Med, 43(5), 657-666. 10.1080/10790268.2019.1671076 Fox, S., Aranko, O., Heilala, J., & Vahala, P. (2020). Exoskeletons Comprehensive, comparative and critical analyses of their potential to improve manufacturing performance. Journal of Manufacturing Technology Management, 31(6), 1261-1280. 10.1108/Jmtm-01-2019-0023 Gonsalves, N., Akanmu, A., Gao, X. H., Agee, P., & Shojaei, A. (2023). Industry Perception of the Suitability of Wearable Robot for Construction Work. Journal of Construction Engineering and Management, 149(5). 10.1061/JCEMD4.COENG-12762 Gonsalves, N. J., Ogunseiju, O. R., Akanmu, A. A., & Nnaji, C. A. (2021). Assessment of a Passive Wearable Robot for Reducing Low Back Disorders during Rebar Work. Journal of Information Technology in Construction, 26, 936-952. 10.36680/j.itcon.2021.050 Greco, A., Caterino, M., Fera, M., & Gerbino, S. (2020). Digital Twin for Monitoring Ergonomics during Manufacturing Production. Applied Sciences-Basel, 10(21). 10.3390/app10217758 Howard, J., Murashov, V. V., Lowe, B. D., & Lu, M. L. (2020). Industrial exoskeletons: Need for intervention effectiveness research. American Journal of Industrial Medicine, 63(3), 201-208. 10.1002/ajim.23080 Karim, F., Majumdar, S., & Darabi, H. (2019). Insights into LSTM fully convolutional networks for time series classification. Ieee Access, 7, 67718-67725. 10.1109/ACCESS.2019.2916828 Kim, S., Moore, A., Srinivasan, D., Akanmu, A., Barr, A., Harris-Adamson, C., Rempel, D. M., & Nussbaum, M. A. (2019). Potential of Exoskeleton Technologies to Enhance Safety, Health, and Performance in Construction: Industry Perspectives and Future Research Directions. Iise Transactions on Occupational Ergonomics & Human Factors, 7(3-4), 185-191. 10.1080/24725838.2018.1561557 Locklin, A., Jung, T., Jazdi, N., Ruppert, T., & Weyrich, M. (2021). Architecture of a Human-Digital Twin as Common Interface for Operator 4.0 Applications. Procedia CIRP, 104, 458–463. 10.1016/j.procir.2021.11.077. Mahmud, D., Bennett, S. T., Zhu, Z. H., Adamczyk, P. G., Wehner, M., Veeramani, D., & Dai, F. (2022). Identifying Facilitators, Barriers, and Potential Solutions of Adopting Exoskeletons and Exosuits in Construction Workplaces. Sensors, 22(24). 10.3390/s22249987 Man, S. S., Nordin, M., Cheng, M. C., Fan, S. M., Lee, S. Y., Wong, W. S., & So, B. C. L. (2022). Effects of passive exoskeleton on trunk and gluteal muscle activity, spinal and hip kinematics and perceived exertion for physiotherapists in a simulated chair transfer task: A feasibility study. International Journal of Industrial Ergonomics, 90, 103323. 10.1016/j.ergon.2022.10332 Massardi, S., Pinto-Fernandez, D., Babic, J., Dezman, M., Trost, A., Grosu, V., Lefeber, D., Rodriguez, C., Bessler, J., Schaake, L., Prange-Lasonder, G., Veneman, J. F., & Torricelli, D. (2023). Relevance of hazards in exoskeleton applications: a survey-based enquiry. J Neuroeng Rehabil, 20(1), 68. 10.1186/s12984-023-01191-y. Moyon, A., Poirson, E., & Petiot, J. F. (2018). Experimental study of the physical impact of a passive exoskeleton on manual sanding operations. 28th Cirp Design Conference 2018, 70, 284-289. 10.1016/j.procir.2018.04.028. Ogunseiju, O., Olayiwola, J., Akanmu, A., & Olatunji, O. A. (2022). Evaluation of postural-assist exoskeleton for manual material handling. Engineering Construction and Architectural Management, 29(3), 1358-1375. 10.1108/Ecam-07-2020-0491 Ogunseiju, O., Olayiwola, J., Akanmu, A. A., & Nnaji, C. (2021). Digital twin-driven framework for improving self-management of ergonomic risks. Smart and Sustainable Built Environment, 10(3), 403-419. 10.1108/Sasbe-03-2021-0035. Omoniyi, A., Trask, C., Milosavljevic, S., & Thamsuwan, O. (2020). Farmers' perceptions of exoskeleton use on farms: Finding the right tool for the work(er). International Journal of Industrial Ergonomics, 80. 10.1016/j.ergon.2020.103036. Picchiotti, M. T., Weston, E. B., Knapik, G. G., Dufour, J. S., & Marras, W. S. (2019). Impact of two postural assist exoskeletons on biomechanical loading of the lumbar spine. Applied Ergonomics, 75, 1-7. 10.1016/j.apergo.2018.09.006. Sharotry, A., Jimenez, J. A., Mediavilla, F. A. M., Wierschem, D., Koldenhoven, R. M., & Valles, D. (2022). Manufacturing Operator Ergonomics: A Conceptual Digital Twin Approach to Detect Biomechanical Fatigue. Ieee Access, 10, 12774-12791. 10.1109/Access.2022.3145984. Sharotry, A., Jimenez, J. A., Wierschem, D., Mediavilla, F. A. M., Koldenhoven, R. M., Valles, D., Koutitas, G., & Aslan, S. (2020). A Digital Twin Framework for Real-Time Analysis and Feedback of Repetitive Work in the Manual Material Handling Industry. 2020 Winter Simulation Conference (Wsc), 2637-2648. 10.1109/Wsc48552.2020.9384043. Siedl, S. M., Wolf, M., & Mara, M. (2021). Exoskeletons in the Supermarket: Influences of Comfort, Strain Relief and Task-Technology Fit on Retail Workers' Post-Trial Intention to Use. Hri '21: Companion of the 2021 Acm/Ieee International Conference on Human-Robot Interaction, 397-401. 10.1145/3434074.3447200. Sowjanya, A. M., & Mrudula, O. (2023). Effective treatment of imbalanced datasets in health care using modified SMOTE coupled with stacked deep learning algorithms. Applied Nanoscience, 13(3), 1829-1840. 10.1007/s13204-021-02063-4. Theurel, J., Desbrosses, K., Roux, T., & Savescu, A. (2018). Physiological consequences of using an upper limb exoskeleton during manual handling tasks. Applied Ergonomics, 67, 211-217. 10.1016/j.apergo.2017.10.008. Upasani, S., Franco, R., Niewolny, K., & Srinivasan, D. (2019). The Potential for Exoskeletons to Improve Health and Safety in Agriculture-Perspectives from Service Providers. Iise Transactions on Occupational Ergonomics & Human Factors, 7(3-4), 222-229. 10.1080/24725838.2019.1575930. Xiong, W. Y., Xu, X. A., Chen, L., & Yang, J. (2022). Sound-Based Construction Activity Monitoring with Deep Learning. Buildings, 12(11). 10.3390/buildings12111947. Zhang, N., Bahsoon, R., Tziritas, N., & Theodoropoulos, G. (2022). Explainable Human-in-the-loop Dynamic Data-Driven Digital Twins. ArXiv. 10.48550/arXiv.2207.09106. Zhang, Y., Xie, X., Li, H., Zhou, B., Wang, Q., & Shahrour, I. (2022). Subway tunnel damage detection based on in-service train dynamic response, variational mode decomposition, convolutional neural networks and long short-term memory. Automation in Construction, 139, 104293. 10.1016/j.autcon.2022.104293