Validation of psychometric instruments in social and health sciences: a practical guide

• López-Pina, Jose-Antonio ^[1] ; Veas, Alejandro ^[1]
  1. [1] Universidad de Murcia

     Universidad de Murcia

     Murcia, España

     
• Localización: Anales de psicología, ISSN-e 1695-2294, ISSN 0212-9728, Vol. 40, Nº. 1, 2024 (Ejemplar dedicado a: January - April), págs. 163-170
• Idioma: inglés
• DOI: 10.6018/analesps.583991
• Títulos paralelos:
  • Validación de instrumentos psicométricos en ciencias sociales y de la salud: una guía práctica
• Enlaces
  • Texto completo
• Dialnet Métricas: 2 Citas
• Resumen
  • español

    Recientemente se ha incrementado significativamente el número de estudios psicométricos junto a avances estadísticos cruciales para evaluar la fiabilidad y validez de los tests. Dada la importancia de proporcionar procedimientos más exactos tanto en la metodología como en la interpretación de las puntuaciones, los editores de la revista Anales de Psicología proponen esta guía para abordar los tópicos más relevantes en el campo de la psicometría aplicada. Con esta finalidad, el presente manuscrito analiza los tópicos principales de la Teoría Clásica de Tests (e.g., análisis factorial exploratorio/confirmatorio, fiabilidad, validez, sesgo, etc.) con vistas a sintetizar y clarificar las aplicaciones prácticas, y mejorar los estándares de publicación de estos trabajos.

  • English

    In recent years there has been a significant rise in the number of psychometric studies, together with crucial statistical advances for validity and reliability measures. Given the importance of providing accurate procedures both in methodology and score interpretation of tests and/or measurement scales, the editors-in-chief of the journal Annals of Psychology have drafted this guide to address the most relevant issues in the field of applied psychometry. To this end, the present manuscript analyses the main topics under the Classical Test Theory framework (e.g., exploratory/confirmatory factor analysis; reliability, validity, bias, etc.) aiming to synthesize and clarify the best practical applications, and improve publication standards.

     

• Referencias bibliográficas
  • Abad, F. J., Olea, J., Ponsoda, V., & García, C. (2011). Medición en ciencias sociales y de la salud [Measurement in social and health...
  • Adams, R. J., Wu, M. L., Cloney, D., Berezner, A., & Wilson, M. (2020). ACER ConQuest: Generalised Item Response Modelling Software (Version...
  • Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43, 561-573. https://doi.org/10.1007/BF02293814
  • Andrich, D., & Luo, G. (1996). RUMMFOLDss: A Windows program for analyzing single stimulus responses of persons to items according to...
  • American Educational Research Association. American Psychological Association. National Council on Measurement in Education (2014). Standards...
  • Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling, 16, 397-438. https://doi.org/10.1080/10705510903008204
  • Beauducel, A., & Herzberg, P. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares...
  • Bentler, P. M., & Yuan, K. H. (1999). Structural equation modeling with small samples: Test statistics. Multivariate Behavioral Research,...
  • Bock, R. D., & Gibbons, R. (2010). Factor analysis of categorical item responses. In M. L. Nering and R. Ostini (Eds.). Handbook of polytomous...
  • Bond, T. G., & Fox, C. (2015). Applying the Rasch model; fundamental measurement in the Human Sciences. Routledge.
  • Brown, T. A. (2006). Confirmatory factor analysis for applied research. The Guilford Press.
  • Byrne, B. M., Shavelson, R. J. & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of...
  • Canivez, G. L. (2016). Bifactor modeling. In K. Schweizer & C. DiStefano (Eds), Principles and methods of test construction (pp. 247-271)....
  • Charter, R. A. (2000). Confidence interval formulas for split-half reliability coefficients. Psychological Reports, 86, 1168-1170. https://doi.org/10.1177/003329410008600317.2
  • Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98-104. https://doi.org/10.1037/0021-9010.78.1.98
  • Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. Holt, Rinehart, and Winston.
  • de Ayala, R. J. (2009). The theory and practice of item response theory. The Guilford Press.
  • de Boeck, P., & Wilson, M. (Eds.) (2004). Explanatory item response models: A generalized linear and nonlinear approach. Springer-Verlag.
  • Enders, C. K. (2004). The impact of missing data on sample reliability estimates: Implications for reliability reporting practices. Educational...
  • Feldt, L. S., & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (pp. 105–146). Macmillan Publishing Co,...
  • Ferrando, P. J., & Lorezo-Seva, U. (2014). Exploratory item factor analysis: additional considerations. Annals of Psychology, 30(3), 1170-1175....
  • Finney, S. J. & DiStefano, C. (2006). Nonnormal and categorical data in structural equation models. In G. R. Hancock, & R. O. Mueller...
  • Fisher, G. H., & Molenaar, I. W. (Eds.) (1995). Rasch models: Foundations, recent developments, and applications. Springer-Verlag.
  • Flora, D. B., & Curran, P. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with...
  • Forero, C., Maydeu-Olivares, A., & Gallardo-Pujol, D. (2009). Factor analysis with ordinal indicators: A Monte Carlo study comparing DWLS...
  • Gilmer, J. S., & Feldt, L. S. (1983). Reliability estimation for a test with part of unknown lengths. Psychometrika, 48, 99-111. https://doi.org/10.1007/BF02314679
  • Goretzko, D., Pham, T. T. H., & Bühner, M. (2021). Exploratory factor analysis: Current use, methodological developments, and recommendations...
  • Green, S. B., Lissitz, R. W., & Mulaik, S. A. (1977). Limitations of coefficient alpha as an index of test unidimensionality. Educational...
  • Hambleton, R. K., Merenda, P. F., & Spielberger, C. D. (2005). Adapting educational and psychological tests for cross-cultural assessment....
  • Henson, R. K. (2001). Understanding internal consistency reliability estimates: A conceptual primer on coefficient alpha. Measurement and...
  • Jackson, D. L. (2001). Sample size and number of parameter estimates in maximum likelihood confirmatory factor analysis: A Monte Carlo investigation....
  • Lei, P. W. (2009). Evaluating estimation methods for ordinal data in structural equation modeling. Quality and Quantity, 43, 495-507. https://doi.org/10.1007/s11135-007-9133-z
  • Linacre, J.M. (2023). Winsteps® (Version 5.6.0) [Computer Software]. Portland, Oregon: Winsteps.com. Available from https://www.winsteps.com/
  • Lloret, S., Ferreres, A., Hernández, A., & Tomás, I. (2014). Exploratory item factor analysis: A practical guide revised and updated....
  • Lloret, S., Ferreres, A., Hernández, A., & Tomás, I. (2017). The exploratory factor analysis of items: guided analysis based on empirical...
  • Lohr, K. N., Aaronson, N. K., Alonso, J., Burnam, M. A., Patrick, D. L., Perrin, E. B., & Roberts, J. S. (1996). Evaluating quality-of-life...
  • Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
  • Masters, G. (1982). A Rasch model for credit partial scoring. Psychometrika, 47, 149-174. https://doi.org/10.1007/BF02296272
  • McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: LEA.
  • McHorney, C. A., & Tarlov, A. R. (1995). Individual-patient monitoring in clinical practice: Are available health status surveys adequate?...
  • Mearns, J., Patchett, E., & Catanzaro, S. (2009). Multitrait-multimethod matrix validation of the Negative Mood Regulation Scale. Journal...
  • Meyer, J. P. (2014). Applied measurement with jMetrik. Routdlege.
  • Michell, J. (1999). Measurement in Psychology: A critical history of a methodological concept. Cambridge University Press.
  • Millsap, R. E. (2011). Statistical approaches to measurement invariance. Routledge.
  • Millsap, R. E., & Yun-Tein, J. (2004). Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research,...
  • Muñiz, J., & Bartram, D. (2007). Improving international tests and testing. European Psychologist, 12, 206-219. https://doi.org/10.1027/1016-9040.12.3.206
  • Muraki, E. (1990). Fitting a polytomous item response model to Likert-type data. Applied Psychological Measurement, 14, 59-71. https://doi.org/10.1177/014662169001400106
  • Nering, M. L., & Ostini, R. (2010). Handbook of polytomous item response theory models. New York: Routledge.
  • O'Rourke, N. (2004). Reliability generalization of responses by care providers to the Center for Epidemiologic Studies-Depression Scale....
  • Raykov, T. (2001). Estimation of congeneric scale reliability using covariance structure analysis with nonlinear restrictions. British Journal...
  • Raykov, T. (2002). Analytic estimation of standard error and confidence interval for scale reliability. Multivariate Behavioral Research,...
  • Raykov, T. (2004). Behavioral scale reliability and measurement invariance evaluation using latent variable modeling. Behavior Therapy, 35,...
  • Reise, S. P., Morizot, J., & Hays, R. D. (2007). The role of the bifactor model in resolving dimensionality issues in health outcome measures....
  • Robitzsch, A., & Lüdtke, O. (2023). Why full, partial, or approximate measurement invariance are not a prerequisite for meaningful and...
  • Rutkowski, L., & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the context of large-scale international surveys....
  • Samejima, E. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, No. 17.
  • Sánchez-Meca, J., Marín-Martínez, F., López-López, J. A., Núñez-Núñez, R. M., Rubio-Aparicio, M., López-García, J. J., López-Pina, J. A.,...
  • Schmitt, N. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8, 350-353. https://doi.org/10.1037/1040-3590.8.4.350
  • Schmitt, N., & Kuljanin, G. (2008). Mesurement invariance: Review of practice and implications. Human Resource Management Review, 18(4),...
  • Shevlin, M., Miles, J. N. V., Davies, M. N. O., & Walker, S. (2000). Coefficient alpha: A useful indicator of reliability? Personality...
  • Sijtsma, K., & Molenaar, I. W. (2002). Introduction to nonparametric item response theory. Sage.
  • Streiner, D. L. (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment,...
  • Streiner, D., Norman, G., & Cairney, J. (2015). Health measurement scales: A practical guide to their development and use. Oxford.
  • Svetina, D., Rutkowski, I., & Rutkowski, D. (2020). Multiple-group invariance with categorical outomes using updated guidelines: an illustration...
  • The jamovi project (2023). jamovi (Version 2.3) [Computer Software]. Retrieved from https://www.jamovi.org
  • Thompson, B., & Vacha-Haase, T. (2000). Psychometrics is datametrics: The test is not reliable. Educational and Psychological Measurement,...
  • Thompson, M. S. (2016). Assessing measurement invariance of scales using Multiple-Group Structural Equation Modeling. In K. Schewizer &...
  • Vacha-Haase, T. (1998). Reliability generalization: Exploring variance in measurement error affecting score reliability across studies. Educational...
  • van der Linden, W., & Hambleton, R. K. (Eds.) (1997). Handbook of modern item response theory. Springer.
  • Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and...
  • Viladrich, C., Angulo-Brunet, A., & Doval, E. (2017). A journey around alpha and omega to estimate internal consistency reliability. Annals...
  • Wright, B. D., & Stone, M. H. (1979). Best test design. Mesa Press.
  • Wright, B. D., & Masters, G. N. (1982). Rating scale analysis. Mesa Press.
  • Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005). Cronbach’s α, Revelle's β, and McDonald's ωH: Their relations with each...
  • Zinbarg, R. E., Yovel, I., Revelle, W., & McDonald, R. P. (2006). Estimating generalizability to a latent variable common to all of a...