Multipoint vs slider: a protocol for experiments Venera Tomaselli University of Catania, Italy Giulio Giacomo Cantone University of Catania, Italy This is a section of ASA 2021 Statistics and Information Systems for Policy Evaluation(DOI: 10.36253/978-88-5518-304-8) by Bruno Bertaccini, Luigi Fabbris, Alessandra Petrucci Firenze University Press Firenze 2021 https://doi.org/10.36253/978-88-5518-304-8.19

Available for academic research purposes

Open Access

Copyright Author(s)

Content licence CC BY 4.0

Metadata licence CC0 1.0

This is original content, published for academic research purposes

 Digital edition XML powered by Booksflow

Since the broad diffusion of Computer-Assisted survey tools (i.e. web surveys), a lively debate about innovative scales of measure arose among social scientists and practitioners. Implications are relevant for applied Statistics and evaluation research since while traditional scales collect ordinal observations, data from sliders can be interpreted as continuous. Literature, however, report excessive times of completion of the task from sliders in web surveys. This experimental protocol is aimed at testing hypotheses on the accuracy in prediction and dispersion of estimates from anonymous participants who are recruited online and randomly assigned into tasks in recognition of shades of colour. The treatment variable is two scales: a traditional multipoint 0-10 multipoint vs a slider 0-100. Shades have a unique parametrisation (true value) and participants have to guess the true value through the scale. These tasks are designed to recreate situations of uncertainty among participants while minimizing the subjective component of a perceptual assessment and maximizing information about scale-driven differences and biases. We propose to test statistical differences in the treatment variable: (i) mean absolute error from the true value (ii), time of completion of the task. To correct biases due to the variance in the number of completed tasks among participants, data about participants can be collected through both pre-tasks acceptance of web cookies and post-tasks explicit questions.

 slider scales colour recognition web-survey design

It is available online at https://doi.org/10.36253/978-88-5518-304-8.19

References Agresti A. (2010). Analysis of Ordinal Categorical Data, Wiley, Hoboken, (NJ). Askalidis, G., Kim, S.J., Malthouse, E.C. (2017). Understanding and overcoming biases in online review systems. Decision Support Systems, 97, pp. 23-30. Austin, P.C., Brunner, L.J. (2003). Type I error inflation in the presence of a ceiling effect. The American Statistician, 57(2), pp. 97-104. Bosch, O.J., Revilla, M., DeCastellarnau, A., Weber, W. (2018). Measurement reliability, validity, and quality of slider versus radio button scales in an online probability-based panel in Norway. Social Science Computer Review. 10.1177/0894439317750089 Chyung, S.Y.Y., Swanson, I., Roberts, K., Hankinson A. (2018). Evidence-based survey design: The use of continuous rating scales in surveys, Performance Improvement, 57(5), 38-48. Couper, M.P., Tourangeau, R., Conrad, F.G., Singer, E. (2006). Evaluating the effectiveness of visual analog scales. Social Science Computer Review, 24(2), pp. 227-245. Finn A., Ranchhod V. (2015). Genuine fakes: The prevalence and implications of data fabrication in a large South African survey. The World Bank Economic Review. 10.1093/wber/lhv054 Fryer, L.K., Nakao, K. (2020). The future of survey self-report: An experiment contrasting Likert, VAS, slide, and swipe touch interfaces. Frontline Learning Research, 8(3), pp. 10-25. Funke, F. (2015) A web experiment showing negative effects of slider scales compared to visual analogue scales and radio button scales, Social Science Computer Review, 34(2), pp. 244-254. Kampen, J., Swyngedouw, M. (2000). The ordinal controversy revisited. Quality & Quantity, 34, pp. 87-102. Kluver, D., Ekstrand, M. D., Konstan, J. A. (2018). Rating-based collaborative filtering: algorithms and evaluation. In Social Information Access, eds. P. Brusilovsky and D. He, Springer, Charm, (SW), pp. 344-390. Liu M., Conrad, F.G. (2018). Where should I start? On default values for slider questions in web surveys, Social Science Computer Review. 10.1177/0894439318755336 Lorenz, J. (2006). Universality in movie rating distributions. The European Physical Journal B. 71, pp. 251-258. Matejka, J., Glueck, M., Grossman, T., Fitzmaurice, G. (2016). The effect of visual appearance on the performance of continuous sliders and visual analogue scales. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 10.1145/2858036.2858063 Roberts, J.M., Brewer, D.D. (2001). Measures and tests of heaping in discrete quantitative distributions. Journal of Applied Statistics, 28(7), pp. 887-896. Roster, C.A., Lucianetti L., Albaum, G. (2015). Exploring slider vs. categorical response formats in web-based surveys, Journal of Research Practice, 11(1), Article D1. Retrieved from http://jrp.icaap.org/index.php/jrp/article/view/509/413. Tomaselli, V., Cantone, G.G. (2020). Evaluating Rank-Coherence of Crowd Rating in Customer Satisfaction. Social Indicators Research. 10.1007/s11205-020-02581-8 Velleman, P.F., Wilkinson, L. (1993). Nominal, ordinal, interval, and ratio typologies are misleading. American Statistician, 47(1), pp. 65-72. Voutilainen, A., Pitkäaho, T., Kvist, T., Vehviläinen-Julkunen, K. (2016). How to ask about patient satisfaction? The visual analogue scale is less vulnerable to confounding factors and ceiling effect than a symmetric Likert scale. Journal of Advanced Nursing, 72(4), pp. 946-957. Zinn, S., Würbach, A. (2015). A statistical approach to address the problem of heaping in self- reported income data. Journal of Applied Statistics, 43(4), pp. 682-703.