To be Direct or not: Reversing Likert Response Format Items

• Jaime García-Fernández ^[1] ; Álvaro Postigo ^[1] ; Marcelino Cuesta ^[1] ; Covadonga González-Nuevo ^[1] ; Álvaro Menéndez-Aller ^[1] ; Eduardo García-Cueto ^[1]
  1. [1] Universidad de Oviedo

     Universidad de Oviedo

     Oviedo, España

     
• Localización: The Spanish Journal of Psychology, ISSN 1138-7416, Nº 25, 2022, págs. 1-9
• Idioma: inglés
• DOI: 10.1017/SJP.2022.20
• Enlaces
  • Texto completo
• Dialnet Métricas: 2 Citas
• Resumen

  • Likert items are often used in social and health sciences. However, the format is strongly affected by acquiescence and reversed items have traditionally been used to control this response bias, a controversial practice. This paper aims to examine how reversed items affect the psychometric properties of a scale. Different versions of the Grit-s scale were applied to an adult sample (N = 1,419). The versions of the scale had either all items in positive or negative forms, or a mix of positive and negative items. The psychometric properties of the different versions (item analysis, dimensionality and reliability) were analyzed. Both negative and positive versions demonstrated better functioning than mixed versions. However, the mean total scores did not vary, which is an example of how similar means could mask other significant differences. Therefore, we advise against using mixed scales, and consider the use of positive or negative versions preferable.

• Referencias bibliográficas
  • Amón Hortelano, J. (1990). Estadística para psicólogos: Estadística descriptiva [Statistics for psychologists: Descriptive statistics] (12th...
  • Arco-Tirado, J. L., Fernández-Martín, F. D., & Hoyle, R. H. (2018). Development and validation of a Spanish version of the Grit-S scale....
  • Areepattamannil, S., & Khine, M. S. (2018). Evaluating the psychometric properties of the original Grit Scale using rasch analysis in...
  • Carlson, M., Wilcox, R., Chou, C.-P., Chang, M., Yang, F., Blanchard, J., Marterella, A., Kuo, A., & Clark, F. (2011). Psychometric properties...
  • Chiavaroli, N. (2017). Negatively-worded multiple choice questions: An avoidable threat to validity. Practical Assessment, Research and Evaluation,...
  • Cordero, A., Pamos, A., & Seisdedos, N. (2008). Inventario de Personalidad Neo Revisado (NEO PI-R), Inventario Neo Reducido de Cinco Factores...
  • Costa, P. T., & McCrae, R. R. (1985). The NEO personality inventory manual. Psychological Assessment Resources.
  • Datu, J. A. D., King, R. B., Valdez, J. P. M., & Eala, M. S. M. (2019). Grit is associated with lower depression via meaning in life among...
  • DiStefano, C., & Motl, R. W. (2009). Personality correlates of method effects due to negatively worded items on the Rosenberg Self-Esteem...
  • Duckworth, A. (2016). Grit: The power of passion and perseverance. Scribner/Simon & Schuster.
  • Duckworth, A. L., & Quinn, P. D. (2009). Development and validation of the Short Grit Scale (Grit–S). Journal of Personality Assessment,...
  • Essau, C. A., Olaya, B., Anastassiou-Hadjicharalambous, X., Pauli, G., Gilvarry, C., Bray, D., O’Callaghan, J., & Ollendick, T. H. (2012)....
  • Feldt, L. S. (1969). A test of the hypothesis that Cronbach’s alpha or Kuder-Richardson coefficient twenty is the same for two tests. Psychometrika,...
  • Ferrando, P. J., & Anguiano-Carrasco, C. (2010). El análisis factorial como técnica de investigación en psicología [Factor analysis as...
  • Ferrando, P. J., & Lorenzo-Seva, U. (2017). Assessing the quality and appropriateness of factor solutions and factor score estimates in...
  • García-Cueto, E. (1994). Coeficiente de congruencia [Congruence coefficient]. Psicothema, 6(3), 465–468.
  • Calderón Garrido, C., Navarro González, D., Lorenzo Seva, U., & Ferrando Piera, P. J. (2019). Multidimensional or essentially unidimensional?...
  • Gonzalez, O., Canning, J. R., Smyth, H., & MacKinnon, D. P. (2020). A psychometric evaluation of the Short Grit Scale. European Journal...
  • Haladyma, T. M., & Rodríguez, M. C. (2013). Developing and validating test items. Taylor & Francis. https://doi.org/10.4324/9780203850381
  • Horan, P. M., DiStefano, C., & Motl, R. W. (2003). Wording effects in self-esteem scales: Methodological artifact or response style? Structural...
  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives....
  • Irwing, P., Booth, T., & Hughes, D. J. (2018). The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale...
  • Józsa, K., & Morgan, G. A. (2017). Reversed items in likert scales: Filtering out invalid responders. Journal of Psychological and Educational...
  • Kaiser, H. F., & Rice, J. (1974). Little Jiffy, Mark Iv. Educational and Psychological Measurement, 34(1), 111–117. https://doi.org/10.1177/001316447403400115
  • Lane, S., Raymond, M. R., & Haladyma, T. M. (2016). Handbook of test development (2nd Rd.). Routledge. https://doi.org/10.4324/9780203102961
  • Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140), 55.
  • Lloret-Segura, S., Ferreres-Traver, A., Hernández-Baeza, A., & Tomás-Marco, I. (2014). El análisis factorial exploratorio de los ítems:...
  • Lorenzo-Seva, U., & Ferrando, P. J. (2013). FACTOR 9.2. Applied Psychological Measurement, 37(6), 497–498. https://doi.org/10.1177/0146621613487794
  • Marsh, H. W. (1986). Negative item bias in ratings scales for preadolescent children: A cognitive-developmental phenomenon. Developmental...
  • Morell, M., Yang, J. S., Gladstone, J. R., Turci Faust, L., Ponnock, A. R., Lim, H. J., & Wigfield, A. (2021). Grit: The long and short...
  • Moreno, R., Martínez, R. J., & Muñiz, J. (2015). Guidelines based on validity criteria for the development of multiple choice items. Psicothema,...
  • Muñiz, J., & Fonseca-Pedrero, E. (2019). Diez pasos para la construcción de un test [Ten steps for test development]. Psicothema, 31(1),...
  • Musumari, P. M., Tangmunkongvorakul, A., Srithanaviboonchai, K., Techasrivichien, T., Suguimoto, S. P., Ono-Kihara, M., & Kihara, M. (2018)....
  • Navarro-González, D., Lorenzo-Seva, U., & Vigil-Colet, A. (2016). Efectos de los sesgos de respuesta en la estructura factorial de los...
  • Ley Orgánica 3/2018, de 5 de diciembre, de Protección de Datos Personales y garantía de los derechos digitales [Organic Law 3/2019, of December...
  • Paulhus, D. L., & Vazire, S. (2005). The self-report method. In Robins, R. W., Fraley, R. C., & Krueger, R. F. (Eds.), Handbook of...
  • Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on...
  • Postigo, Á., Cuesta, M., García-Cueto, E., Menéndez-Aller, Á., González-Nuevo, C., & Muñiz, J. (2021). Grit assessment: Is one dimension...
  • Solís Salazar, M. (2015). The dilemma of combining positive and negative items in scales. Psicothema, 27(2), 192–199. https://doi.org/10.7334/psicothema2014.266
  • Suárez-Álvarez, J., Pedrosa, I., Lozano, L. M., García-Cueto, E., Cuesta, M., & Muñiz, J. (2018). Using reversed items in Likert scales:...
  • Timmerman, M. E., & Lorenzo-Seva, U. (2011). Dimensionality assessment of ordered polytomous items with parallel analysis. Psychological...
  • van Sonderen, E., Sanderman, R., & Coyne, J. C. (2013). Ineffectiveness of reverse wording of questionnaire items: Let’s learn from cows...
  • Vazsonyi, A. T., Ksinan, A. J., Ksinan Jiskrova, G., Mikuška, J., Javakhishvili, M., & Cui, G. (2019). To grit or not to grit, that is...
  • Vigil-Colet, A., Navarro-González, D., & Morales-Vives, F. (2020). To reverse or to not reverse likert-type items: That is the question....
  • Wong, N., Rindfleisch, A., & Burroughs, J. E. (2003). Do reverse-worded items confound measures in cross-cultural consumer research? The...
  • Woods, C. M. (2006). Careless responding to reverse-worded items: Implications for confirmatory factor analysis. Journal of Psychopathology...