Cognitive Dynamics for Construction Management Learning Tasks in Mixed Reality Environments Xuanchang Liu Illinois Institute of Technology, United States Ivan Mutis Illinois Institute of Technology, United States This is a section of CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality (DOI: 10.36253/979-12-215-0289-3) by Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi Firenze University Press Florence 2023 https://doi.org/10.36253/10.36253/979-12-215-0289-3.22

Available for academic research purposes

Open Access

Copyright Author(s)

Content licence CC BY-NC 4.0

Metadata licence CC0 1.0

This is original content, published for academic research purposes

 Digital edition XML powered by Booksflow

Technologies to communicate construction project information (engineering designs, schedules) have evolved into a wider range of innovative ecosystems for engineering practices (e.g., cloud-based 3D representations and advanced immersive environments). There is a lack of exploration of effective user interaction for learning and training in relation to how presented information influences cognition in these ecosystems. The presented research investigates the users’ cognitive and attentional differences using the interactive capabilities of Mixed reality (MX) technology. The enhanced user-situation interactions are analyzed by measuring cognitive dynamics with an emphasis on two processes (attentional focus and cognitive load) in relation to the challenge of the engineering learning task— defined by its complexity (limited time frame for observations of the situations, number of required observations) and nature (episodic). Cognitive dynamics were measured using an electroencephalography (EEG) device that senses electrical activity in response to changing levels of cognitive stimuli via electrodes placed on the scalp. Measuring fluctuations in cognitive processing (related to the intensity of various task demands) allows associating efforts on semantic information processing for learning and training tasks (e.g., walkthroughs for safety checks in job site in MX). The approach enhances opportunities to design technology that best adapts to the user needs for engineering practices with an efficient comprehensive performance assessment

 Electroencephalography (EEG) Dynamics of attention Cognitive load Cognitive processing

It is available online at https://doi.org/10.36253/10.36253/979-12-215-0289-3.22

References Asish, S. M., Kulshreshth, A. K., & Borst, C. W. (2022). Detecting distracted students in educational VR environments using machine learning on eye gaze data. Computers & Graphics, 109, 75-87. 10.1016/J.CAG.2022.10.007 Borghini, G., Vecchiato, G., Toppi, J., Astolfi, L., Maglione, A., Isabella, R., Caltagirone, C., Kong, W., Wei, D., Zhou, Z., Polidori, L., Vitiello, S., & Babiloni, F. (2012). Assessment of mental fatigue during car driving by using high resolution EEG activity and neurophysiologic indices. Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Annual International Conference, 2012, 6442-6445. 10.1109/EMBC.2012.6347469 Chen, J., Song, X., & Lin, Z. (2016). Revealing the “Invisible Gorilla” in construction: Estimating construction safety through mental workload assessment. Automation in Construction, 63, 173-183. 10.1016/J.AUTCON.2015.12.018 Delorme, A., & Makeig, S. (2004). EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods, 134(1), 9-21. 10.1016/J.JNEUMETH.2003.10.009 Derbali, L., & Frasson, C. (2011). Physiological evaluation of attention getting strategies during serious game play. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6738 LNAI, 447-449. 10.1007/978-3-642-21869-9_65/COVER Fernandez-Duque, D., Baird, J. A., & Posner, M. I. (2000). Executive Attention and Metacognitive Regulation. Consciousness and Cognition, 9(2), 288-307. 10.1006/CCOG.2000.0447 Fernandez Rojas, R., Debie, E., Fidock, J., Barlow, M., Kasmarik, K., Anavatti, S., Garratt, M., & Abbass, H. (2020). Electroencephalographic Workload Indicators During Teleoperation of an Unmanned Aerial Vehicle Shepherding a Swarm of Unmanned Ground Vehicles in Contested Environments. Frontiers in Neuroscience, 14, 509340-509340. 10.3389/FNINS.2020.00040/BIBTEX Frank Moore, H., & Gheisari, M. (2019). A Review of Virtual and Mixed Reality Applications in Construction Safety Literature. Safety 2019, Vol. 5, Page 51, 5(3), 51-51. 10.3390/SAFETY5030051 Ghasemy, H., Momtazpour, M., & Sardouie, S. H. (2019). Detection of Sustained Auditory Attention in Students with Visual Impairment. ICEE 2019 - 27th Iranian Conference on Electrical Engineering, 1798-1801. 10.1109/IRANIANCEE.2019.8786565 Guo, H., Yu, Y., & Skitmore, M. (2017). Visualization technology-based construction safety management: A review. Automation in Construction, 73, 135-144. 10.1016/J.AUTCON.2016.10.004 Hillard, B., El-Baz, A. S., Sears, L., Tasman, A., & Sokhadze, E. M. (2013). Neurofeedback training aimed to improve focused attention and alertness in children with ADHD: a study of relative power of EEG rhythms using custom-made software application. Clinical EEG and neuroscience, 44(3), 193-202. 10.1177/1550059412458262 Homan, R. W., Herman, J., & Purdy, P. (1987). Cerebral location of international 10–20 system electrode placement. Electroencephalography and Clinical Neurophysiology, 66(4), 376-382. 10.1016/0013-4694(87)90206-9 Huang, X., Hinze, J., & Asce, M. (2003). Analysis of Construction Worker Fall Accidents. Journal of Construction Engineering and Management, 129(3), 262-271. 10.1061/(ASCE)0733-9364(2003)129:3(262) Hwang, S., Jebelli, H., Choi, B., Choi, M., & Lee, S. (2018). Measuring Workers’ Emotional State during Construction Tasks Using Wearable EEG. Journal of Construction Engineering and Management, 144(7), 04018050-04018050. 10.1061/(ASCE)CO.1943-7862.0001506/ASSET/39CC5604-9C75-4146-8C0A-F7A212977F86/ASSETS/IMAGES/LARGE/FIGURE8.JPG James, W. (1890). The Principles of Psychology. Henry Holt. 10.1037/11059-000 Jebelli, H., Hwang, S., & Lee, S. H. (2018). EEG-based workers' stress recognition at construction sites. Automation in Construction, 93, 315-324. 10.1016/J.AUTCON.2018.05.027 Jeelani, I., Han, K., & Albert, A. (2020). Development of virtual reality and stereo-panoramic environments for construction safety training. Engineering, Construction and Architectural Management, 27(8), 1853-1876. 10.1108/ECAM-07-2019-0391/FULL/XML Kaushik, P., Moye, A., Vugt, M. v., & Roy, P. P. (2022). Decoding the cognitive states of attention and distraction in a real-life setting using EEG. Scientific Reports 2022 12:1, 12(1), 1-10. 10.1038/s41598-022-24417-w Ke, J., Zhang, M., Luo, X., & Chen, J. (2021). Monitoring distraction of construction workers caused by noise using a wearable Electroencephalography (EEG) device. Automation in Construction, 125, 103598-103598. 10.1016/J.AUTCON.2021.103598 Klimesch, W. (1999). EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Research Reviews, 29(2-3), 169-195. 10.1016/S0165-0173(98)00056-3 Mapelli, I., & Özkurt, T. E. (2019). Brain Oscillatory Correlates of Visual Short-Term Memory Errors. Frontiers in human neuroscience, 13. 10.3389/FNHUM.2019.00033 Sacks, R., Perlman, A., & Barak, R. (2013). Construction safety training using immersive virtual reality. 10.1080/01446193.2013.828844 Saedi, S., Fini, A. A. F., Khanzadi, M., Wong, J., Sheikhkhoshkar, M., & Banaei, M. (2022). Applications of electroencephalography in construction. Automation in Construction, 133, 103985-103985. 10.1016/J.AUTCON.2021.103985 Sanei, S., & Chambers, J. (2007). EEG signal processing. 289-289. https://books.google.com/books/about/EEG_Signal_Processing.html?id=f44hLefOz6UC Sarailoo, R., Latifzadeh, K., Amiri, S. H., Bosaghzadeh, A., & Ebrahimpour, R. (2022). Assessment of instantaneous cognitive load imposed by educational multimedia using electroencephalography signals. Frontiers in Neuroscience, 16, 744737-744737. 10.3389/FNINS.2022.744737/BIBTEX Tehrani, B. M., Wang, J., & Truax, D. (2022). Assessment of mental fatigue using electroencephalography (EEG) and virtual reality (VR) for construction fall hazard prevention. Engineering, Construction and Architectural Management, 29(9), 3593-3616. 10.1108/ECAM-01-2021-0017/FULL/XML Top 10 Most Frequently Cited Standards | Occupational Safety and Health Administration. In. Ultracortex Mark IV | OpenBCI Documentation. In. Wilkins, J. R. (2011). Construction workers' perceptions of health and safety training programmes. Construction Management and Economics, 29(10), 1017-1026. 10.1080/01446193.2011.633538 Winkler, I., Brandl, S., Horn, F., Waldburger, E., Allefeld, C., & Tangermann, M. (2014). Robust artifactual independent component classification for BCI practitioners. Journal of neural engineering, 11(3). 10.1088/1741-2560/11/3/035013 Zietsch, B. P., Hansen, J. L., Hansell, N. K., Geffen, G. M., Martin, N. G., & Wright, M. J. (2007). Common and specific genetic influences on EEG power bands delta, theta, alpha, and beta. Biological Psychology, 75(2), 154-164. 10.1016/J.BIOPSYCHO.2007.01.004 Zou, Z., & Ergan, S. (2023). Towards emotionally intelligent buildings: A Convolutional neural network based approach to classify human emotional experience in virtual built environments. Advanced Engineering Informatics, 55, 101868-101868. 10.1016/J.AEI.2022.101868