Application of Smart Technologies for Assessing Users’ Well-Being for Immersive Design Strategies: A State-of-the-Art Review Eleonora D'Ascenzi University of Florence, Italy Vito Getuli University of Florence, Italy Irene Fiesoli University of Florence, Italy This is a section of CONVR 2023 - Proceedings of the 23rd International Conference on Construction Applications of Virtual Reality (DOI: 10.36253/979-12-215-0289-3) by Pietro Capone, Vito Getuli, Farzad Pour Rahimian, Nashwan Dawood, Alessandro Bruttini, Tommaso Sorbi Firenze University Press Florence 2023 https://doi.org/10.36253/10.36253/979-12-215-0289-3.09

Available for academic research purposes

Open Access

Copyright Author(s)

Content licence CC BY-NC 4.0

Metadata licence CC0 1.0

This is original content, published for academic research purposes

 Digital edition XML powered by Booksflow

As never before, during the COVID-19 pandemic, the effectiveness of the digital design strategies on the user’s well-being has been questioned. However, a research branch astride digital design and neuroscience able to overcome net discipline borders to analyse users’ well-being seems to be lacking. Today mainly qualitative data are used in the design field for the investigation of users’ quality experience. Although fundamental, they also have great disadvantages such as unanswered questions, unconscientious responses, and respondents’ biases. As such, a systematic state of art review is presented to find methodologies and tools currently used in medicine to identify the impact of digital design strategies (XR) on users’ well-being through quantitative and objective data. The main technologies used for this purpose have been synthesized in a schematic chart by reporting the principal related biometric data (skin conductivity, heart rate metrics and breathing rates), as well as other technologies such as video/images/audio analysis based on sensors and machine learning to reach out mass numbers. In conclusion, gaps and future applications of this innovative approach within the virtual environment have been identified by the authors

 extended reality virtual reality neuro-design digital design immersive experience user experience well-being assessment

It is available online at https://doi.org/10.36253/10.36253/979-12-215-0289-3.09

References Abburi, H., Shrivastava, M., & Gangashetty, S. V. (2016). Improved Multimodal Sentiment Detection Using Stressed Regions of Audio. PROCEEDINGS OF THE 2016 IEEE REGION 10 CONFERENCE (TENCON), 2834–2837. 10.1109/TENCON.2016.7848560 Acerbi, G., Rovini, E., Betti, S., Tirri, A., Ronai, J. F., Sirianni, A., Agrimi, J., Eusebi, L., & Cavallo, F. (2017). A Wearable System for Stress Detection Through Physiological Data Analysis. In Cavallo, F and Marletta, V and Monteriu, A and Siciliano, P (Ed.), AMBIENT ASSISTED LIVING (Vol. 426, pp. 31–50). 10.1007/978-3-319-54283-6\_3 Affanni, A., Bernardini, R., Piras, A., Rinaldo, R., & Zontone, P. (2018). Driver’s stress detection using Skin Potential Response signals. MEASUREMENT, 122, 264–274. 10.1016/j.measurement.2018.03.040 Al Abdi, R. M., Alhitary, A. E., Abdul Hay, E. W., & Al-Bashir, A. K. (2018). Objective detection of chronic stress using physiological parameters. Medical & Biological Engineering & Computing, 56(12), 2273–2286. 10.1007/s11517-018-1854-8 Alraouf, A. A. (2021). The new normal or the forgotten normal: contesting COVID-19 impact on contemporary architecture and urbanism. Archnet-IJAR, 15(1), 167–188. 10.1108/ARCH-10-2020-0249 Amerio, A., Brambilla, A., Morganti, A., Aguglia, A., Bianchi, D., Santi, F., Costantini, L., Odone, A., Costanza, A., Signorelli, C., Serafini, G., Amore, M., & Capolongo, S. (2020). Covid-19 lockdown: Housing built environment’s effects on mental health. International Journal of Environmental Research and Public Health, 17(16), 1–10. 10.3390/ijerph17165973 Anusha, A. S., Sukumaran, P., Sarveswaran, V., Surees Kumar, S., Shyam, A., Akl, T. J., Preejith, S. P., & Sivaprakasam, M. (2020). Electrodermal Activity Based Pre-surgery Stress Detection Using a Wrist Wearable. IEEE Journal of Biomedical and Health Informatics, 24(1), 92–100. 10.1109/JBHI.2019.2893222 Attallah, O. (2020). An Effective Mental Stress State Detection and Evaluation System Using Minimum Number of Frontal Brain Electrodes. Diagnostics (Basel, Switzerland), 10(5). 10.3390/diagnostics10050292 Bin, M. S., Khalifa, O. O., & Saeed, R. A. (2015). Real-Time Personalized Stress Detection from Physiological Signals. In Saeed, RA and Mokhtar, RA (Ed.), 2015 International Conference on Computing, Control, Networking, Electronics and Embedded Systems Engineering (ICCNEEE) (pp. 352–356) 10.1109/ICCNEEE.2015.7381390 Burton, E. J., Mitchell, L., & Stride, C. B. (2011). Good places for ageing in place: Development of objective built environment measures for investigating links with older people’s wellbeing. BMC Public Health, 11. 10.1186/1471-2458-11-839 Can, Y. S., Arnrich, B., & Ersoy, C. (2019). Stress detection in daily life scenarios using smart phones and wearable sensors: A survey. In Journal of Biomedical Informatics (Vol. 92). Academic Press Inc. 10.1016/j.jbi.2019.103139 Debard, G., De Witte, N., Sels, R., Mertens, M., Van Daele, T., & Bonroy, B. (2020). Making Wearable Technology Available for Mental Healthcare through an Online Platform with Stress Detection Algorithms: The Carewear Project. JOURNAL OF SENSORS, 2020. 10.1155/2020/8846077 Delmastro, F., Martino, F. D., & Dolciotti, C. (2020). Cognitive Training and Stress Detection in MCI Frail Older People through Wearable Sensors and Machine Learning. IEEE Access, 8, 65573–65590. 10.1109/ACCESS.2020.2985301 Elzeiny, S., & Qaraqe, M. (2018). Blueprint to Workplace Stress Detection Approaches. 2018 INTERNATIONAL CONFERENCE ON COMPUTER AND APPLICATIONS (ICCA), 407–412. 10.1109/COMAPP.2018.8460293 Feng, Z., Li, N., Feng, L., Chen, D., & Zhu, C. (2021). Leveraging ECG signals and social media for stress detection. BEHAVIOUR \& INFORMATION TECHNOLOGY, 40(2), 116–133. 10.1080/0144929X.2019.1673820 Ghaderi, A., Frounchi, J., & Farnam, A. (2015). Machine Learning-based Signal Processing Using Physiological Signals for Stress Detection. 2015 22ND IRANIAN CONFERENCE ON BIOMEDICAL ENGINEERING (ICBME), 93–98. 10.1109/ICBME.2015.7404123 Gjoreski, M., Gjoreski, H., Lutrek, M., & Gams, M. (2015). Automatic Detection of Perceived Stress in Campus Students Using Smartphones. Proceedings - 2015 International Conference on Intelligent Environments, IE 2015, 132–135. 10.1109/IE.2015.27 Gunawardhane, S. D. W., De Silva, P. M., Kulathunga, D. S. B., & Arunatileka, S. M. K. D. (2013). Non Invasive Human Stress Detection Using Key Stroke Dynamics and Pattern Variations. 2013 INTERNATIONAL CONFERENCE ON ADVANCES IN ICT FOR EMERGING REGIONS (ICTER), 240–247 10.1109/ICTer.2013.6761185 Healy, M., Donovan, R., Walsh, P., & Zheng, H. (2018). A Machine Learning Emotion Detection Platform to Support Affective Well Being. In Zheng, H and Callejas, Z and Griol, D and Wang, H and Hu, X and Schmidt, H and Baumbach, J and Dickerson, J and Zhang, L (Ed.), PROCEEDINGS 2018 IEEE INTERNATIONAL CONFERENCE ON BIOINFORMATICS AND BIOMEDICINE (BIBM) (pp. 2694–2700) 10.1109/BIBM.2018.8621562 Kalas, M. S., & Momin, B. F. (2016). Stress Detection and Reduction using EEG Signals. 2016 INTERNATIONAL CONFERENCE ON ELECTRICAL, ELECTRONICS, AND OPTIMIZATION TECHNIQUES (ICEEOT), 471–475 10.1109/ICEEOT.2016.7755604 Kalimeri, K., & Saitis, C. (2016). Exploring Multimodal Biosignal Features for Stress Detection during Indoor Mobility. In Nakano, YI and Andre, E and Nishida, T and Busso, C and Pelachaud, C (Ed.), ICMI’16: PROCEEDINGS OF THE 18TH ACM INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (pp. 53–60). 10.1145/2993148.2993159 Melone, M. R. S., & Borgo, S. (2020). Rethinking rules and social practices. The design of urban spaces in the post-Covid-19 lockdown. TEMA-JOURNAL OF LAND USE MOBILITY AND ENVIRONMENT, SI, 333–341. 10.6092/1970-9870/6923 Minguillon, J., Perez, E., Lopez-Gordo, M. A., Pelayo, F., & Sanchez-Carrion, M. J. (2018). Portable system for real-time detection of stress level. Sensors (Switzerland), 18(8). 10.3390/s18082504 Mozos, O. M., Sandulescu, V., Andrews, S., Ellis, D., Bellotto, N., Dobrescu, R., & Ferrandez, J. M. (2017). Stress detection using wearable physiological and sociometric sensors. International Journal of Neural Systems, 27(2). 10.1142/S0129065716500416 Pandey, P., Lee, E. K., & Pompili, D. (2016). A Distributed Computing Framework for Real-Time Detection of Stress and of Its Propagation in a Team. IEEE Journal of Biomedical and Health Informatics, 20(6), 1502–1512. 10.1109/JBHI.2015.2477342 Pascoe, M. C., Thompson, D. R., & Ski, C. F. (2017). Yoga, mindfulness-based stress reduction and stress-related physiological measures: A meta-analysis. In Psychoneuroendocrinology (Vol. 86, pp. 152–168). Elsevier Ltd. 10.1016/j.psyneuen.2017.08.008 Qiao, S., Li, X., Zilioli, S., Chen, Z., Deng, H., Pan, J., & Guo, W. (2017). Hair measurements of cortisol, DHEA, and DHEA to cortisol ratio as biomarkers of chronic stress among people living with HIV in China: Known-group validation. PLoS ONE, 12(1). 10.1371/journal.pone.0169827 Rachakonda, L., Mohanty, S. P., Kougianos, E., & Sundaravadivel, P. (2019). Stress-Lysis: A DNN-Integrated Edge Device for Stress Level Detection in the IoMT. IEEE TRANSACTIONS ON CONSUMER ELECTRONICS, 65(4), 474–483. 10.1109/TCE.2019.2940472 Rani, P., Sims, J., Brackin, R., & Sarkar, N. (2002). Online stress detection using psychophysiological signals for implicit human-robot cooperation. ROBOTICA, 20(6), 673–685. 10.1017/S0263574702004484 Reanaree, P., Tananchana, P., Narongwongwathana, W., & Pintavirooj, C. (2016). Stress and Office-Syndrome Detection using EEG, HRV and Hand Movement. 2016 9TH BIOMEDICAL ENGINEERING INTERNATIONAL CONFERENCE (BMEICON) 10.1109/BMEiCON.2016.7859624 Sağbaş, E. A., Korukoglu, S., & Balli, S. (2020). Stress Detection via Keyboard Typing Behaviors by Using Smartphone Sensors and Machine Learning Techniques. Journal of Medical Systems, 44(4). 10.1007/s10916-020-1530-z Sriramprakash, S., Prasanna, V. D., & Murthy, O. V. R. (2017a). Stress Detection in Working People. Procedia Computer Science, 115, 359–366. 10.1016/j.procs.2017.09.090 Sriramprakash, S., Prasanna, V. D., & Murthy, O. V. R. (2017b). Stress Detection in Working People. In G. M. Paul Mulerikkal J. (Ed.), Procedia Computer Science (Vol. 115, pp. 359–366). Elsevier B.V. 10.1016/j.procs.2017.09.090 Vizer, L. M., Zhou, L., & Sears, A. (2009). Automated stress detection using keystroke and linguistic features: An exploratory study. INTERNATIONAL JOURNAL OF HUMAN-COMPUTER STUDIES, 67(10), 870–886. 10.1016/j.ijhcs.2009.07.005 Wells, S., Tremblay, P. F., Flynn, A., Russell, E., Kennedy, J., Rehm, J., Van Uum, S., Koren, G., & Graham, K. (2014). Associations of hair cortisol concentration with self-reported measures of stress and mental health-related factors in a pooled database of diverse community samples. Stress, 17(4), 334–342. 10.3109/10253890.2014.930432 Zalabarria, U., Irigoyen, E., Martinez, R., & Salazar-Ramirez, A. (2017). Detection of Stress Level and Phases by Advanced Physiological Signal Processing Based on Fuzzy Logic. In Grana, M and LopezGuede, JM and Etxaniz, O and Herrero, A and Quintian, H and Corchado, E (Ed.), INTERNATIONAL JOINT CONFERENCE SOCO’16- CISIS’16-ICEUTE’16 (Vol. 527, pp. 301–312). 10.1007/978-3-319-47364-2\_29 Zhang, H., Feng, L., Li, N., Jin, Z., & Cao, L. (2020a). Video-based stress detection through deep learning. Sensors (Switzerland), 20(19), 1–17. 10.3390/s20195552 Zhang, H., Feng, L., Li, N., Jin, Z., & Cao, L. (2020b). Video-Based Stress Detection through Deep Learning. SENSORS, 20(19). 10.3390/s20195552 Zhao, L., Li, Q., Xue, Y., Jia, J., & Feng, L. (2016). A systematic exploration of the micro-blog feature space for teens stress detection. Health Information Science and Systems, 4(1), 3. 10.1186/s13755-016-0016-3