Evidence for Test Validation: a Guide for Practitioners

• Autores: Stephen Sireci, María Isabel Benítez
• Localización: Psicothema, ISSN-e 1886-144X, ISSN 0214-9915, Vol. 35, Nº. 3, 2023, págs. 217-226
• Idioma: inglés
• DOI: 10.7334/psicothema2022.477
• Títulos paralelos:
  • Evidencias Sobre la Validación de los Tests: una Guía Práctica
• Enlaces
  • Texto completo (pdf)
• Dialnet Métricas: 8 Citas
• Resumen
  • español

    La validez es un tema central en la evaluación psicológica y educativa. A pesar de que la literatura disponible recoge numerosos recursos en los que se describe el concepto de validez, las fuentes de evidencia y se aportan sugerencias sobre cómo obtener evidencias de validez, apenas existen guías que proporcionen instrucciones específicas para planificar y desarrollar estudios de validación. Método: El presente artículo describe (a) los principios fundamentales en los que se sustenta la validez de los test, (b) el proceso de validación, y (c) una guía práctica para planificar y recoger evidencias de validez que apoyen el uso de un test para alcanzar el objetivo previsto. Resultados: En primer lugar, se describe el concepto de validez y las fuentes de evidencia, aportando ejemplos específicos donde se abordan cada una de ellas. A continuación, se describe una agenda de validación en la que se enumeran los pasos y tareas necesarios para planificar y completar un estudio de validación. Conclusiones: Finalmente, se discute la relevancia de adoptar una aproximación comprehensiva al abordar estudios de validación

  • English

    Validity is a core topic in educational and psychological assessment. Although there are many available resources describing the concept of validity, sources of validity evidence, and suggestions about how to obtain validity evidence; there is little guidance providing specific instructions for planning and carrying out validation studies. Method: In this paper we describe (a) the fundamental principles underlying test validity, (b) the process of validation, and (c) practical guidance for practitioners to plan and carry out sufficient validity research to support the use of a test for its intended purposes. Results: We first define validity, describe sources of validity evidence, and provide examples where each of these sources are addressed. Then, we describe a validation agenda describing steps and tasks for planning and developing validation studies. Conclusions: Finally, we discuss the importance of addressing validation studies from a comprehensive approach

• Referencias bibliográficas
  • American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014)....
  • American Educational Research Association, American Psychological Association & National Council on Measurement in Education. (2018)....
  • American Psychological Association (2010). Ethical principles of psychologists and code of conduct. Author. https://doi.org/10.1037/ amp0000102
  • Atchison, D., Garet, M. S., Smith, T. M., & Song, M. (2022). The validity of measures of instructional alignment with state standards...
  • Beck, K. (2020). Ensuring content validity of psychological and educational tests--the role of experts. Frontline Learning Research, 8(6),...
  • Benítez, I., Van de Vijver, F., & Padilla, J. L. (2022). A mixed methods approach to the analysis of bias in cross-cultural studies. Sociological...
  • Benítez, I., Padilla, J.L., Hidalgo Montesinos, M. D., & Sireci, S. G. (2016). Using mixed methods to interpret differential item functioning....
  • Bhola, D. S., Impara, J. C., & Buckendahl, C. W. (2003). Aligning tests with states’ content standards: Methods and issues. Educational...
  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin,...
  • Cavalcanti, R. V. A., Junior, H. V. M., de Araújo Pernambuco, L., & de Lima, K. C. (2020). Screening for masticatory disorders in older...
  • Cizek, G. J., Rosenberg, S. L., & Koons, H. H. (2008). Sources of validity evidence for educational and psychological tests. Educational...
  • Cronbach, L. J. (1971). Test Validation. In R.L. Thorndike (Ed.) Educational measurement (2nd ed., pp. 443-507). American Council on Education.
  • Cronbach, L. J. & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-302.
  • De Corte, W., Lievens, F., & Sackett, P. R. (2007). Combining predictors to achieve optimal trade-offs between selection quality and adverse...
  • Dee, T. S., & Jacob. B. (2011). The impact of No Child Left Behind on student achievement. Journal of Policy Analysis and Management,...
  • Dumas, D., Dong, Y., & McNeish, D. (2022). How Fair is my Test? A Ratio Coefficient to Help Represent Consequential Validity. European...
  • Engelhardt, L., & Goldhammer, F. (2019). Validating test score interpretations using time information. Frontiers in Psychology, 10, Article...
  • Ferrando, P. J., Lorenzo Seva, U., Hernández Dorado, A., & Muñiz, J. (2022). Decalogue for the factor analysis of test items. Psicothema,...
  • Georgia Department of Education and Data Recognition Corporation. (2019). Georgia Milestones Assessment System 2019 operational technical...
  • Gómez-Benito, J., Sireci, S., Padilla, J.L., Hidalgo, M. D., & Benítez, I. (2018). Differential item functioning: Beyond validity evidence...
  • International Test Commission & Association of Test Publishers (2022). Guidelines for technology-based assessment. International Test...
  • Irwin, V., De La Rosa, J., Wang, K., Hein, S., Zhang, J., Burr, R., Roberts, A., Barmer, A., Bullock Mann, F., Dilig, R., & Parker, S....
  • Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112, 527–535.
  • Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112, 527–535.
  • Kane, M.T. (2006). Validation. In R. L. Brennan (Ed.), Educational Measurement (4th ed., pp. 17–64). American Council on Education and Praeger.
  • Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1-73.
  • Keenan, J. & Meenan, C. E. (2014). Test differences in diagnosing reading comprehension deficits. Journal of Learning Disabilities, 47,...
  • Lafuente-Martínez, M., Lévêque, O., Benítez, I., Hardebolle, C., & Dehler Zufferey, J. (2022). Assessing computational thinking: Development...
  • Lane, S. (2014). Validity evidence based on testing consequences. Psicothema, 26(1), 127-135. https://doi.org/10.7334/psicothema2013.258
  • Lee, S., & Winke, P. (2018). Young learners’ response processes when taking computerized tasks for speaking assessment. Language Testing,...
  • Luesia, J. F., Sánchez-Martín, M., & Benítez, I. (2021). The effect of personal values on academic achievement. Psychological Test and...
  • Marchant, G. J. & Paulson, S. E. (2005, January 21). The relationship of high school graduation exams to graduation rates and SAT scores....
  • Martone, A., & Sireci, S. G. (2009). Evaluating alignment between curriculum, assessment, and instruction. Review of Educational Research,...
  • Messick, S. (1989). Validity. In R. Linn (Ed.), Educational Measurement, (3rd ed., pp. 13-100). American Council on Education.
  • Mislevy, R. J. (2019). On integrating psychometrics and learning analytics in complex assessments. In H. Jiao, R. W. Lissitz, & A. van...
  • Morris, S. B., & Dunleavy, D. M. (2016). Adverse impact analysis: Understanding data, impact, and risk. Routledge.
  • Muñiz, J., & Fonseca-Pedrero, E. (2019). Ten steps for test development. Psicothema, 31(1), 7-16. https://doi.org/10.7334/psicothema2018.291
  • Newman, D. A., Tang, C., Song, Q. C., & Wee, S. (2022). Dropping the GRE, keeping the GRE, or using GRE-optional admissions? Considering...
  • Newton, P. E., & Shaw, S. D. (2013). Standards for talking and thinking about validity. Psychological Methods, 18, 301–319. https://doi.org/10.1037/...
  • Noble, T., Rosebery, A., Suarez, C., Warren, B., & O’Connor, M. C. (2014). Science assessments and English language learners: Validity...
  • Padilla, J. L., & Benítez, I. (2014). Validity evidence based on response processes. Psicothema, 26(1), 136-144. https://doi.org/10.7334/psicothema2013.259
  • Randall, J. (2021). “Color-neutral” is not a thing: Redefining construct definition and representation through a justice-oriented critical...
  • Reynolds, K. A. & Moncaleano, S. (2021). Digital module 26: Content alignment in standards-based educational assessment. Educational Measurement:...
  • Rios, J., & Wells, C. (2014). Validity evidence based on internal structure. Psicothema, 26(1), 108-116. https://doi.org/10.7334/psicothema2013.260
  • Rushton, P. W., Routhier, F., Miller, W. C., Auger, C., & Lavoie, M. P. (2015). French-Canadian translation of the WheelCon-M (WheelCon-MF)...
  • Russell, M. (2022). Clarifying the terminology of validity and the investigative tages of validation. Educational Measurement: Issues and...
  • Segool, N. K., Carlson, J. S., Goforth, A. N., von der Embse, N. & Barterian, J. N. (2013). Heightened test anxiety among young children:...
  • Shepard, L. A. (1993). Evaluating test validity. Review of Research in Education, 19, 405-450.
  • Sinha, R., Oswald, F., Imus, A & Schmitt, N. (2011). Criterion-focused approach to reducing adverse impact in college admissions. Applied...
  • Sireci, S. G. (1998). The construct of content validity. Social Indicators Research, 45(1), 83-117.
  • Sireci, S. G. (2016a). Comments on valid (and invalid?) commentaries. Assessment in Education: Principles, Policy & Practice, 23, 319-321....
  • Sireci, S. G. (2016b). On the validity of useless tests. Assessment in Education: Principles, Policy & Practice, 23, 226-235. https://doi.org/10...
  • Sireci, S. G. (2020). De-“constructing” test validation. Chinese/English Journal of Educational Measurement and Evaluation| 教育测量与评估双语季刊, 1(1),...
  • Sireci, S., & Faulkner-Bond, M. (2014). Validity evidence based on test content. Psicothema, 26(1), 100-107. https://doi.org/10.7334/...
  • Sireci, S. G., & Geisinger, K. F. (1998). Equity issues in employment testing. In J.H. Sandoval, C. Frisby, K. F. Geisinger, J. Scheuneman,...
  • Sireci, S. G., & Greiff, S. (2019). On the importance of educational tests. European Journal of Psychological Assessment, 35, 297-300....
  • Sireci, S. G., Lim, H., Rodriguez, G., Banda, E., & Zenisky, A. (2018, April 12-16). Evaluating criteria for validity evidence based on...
  • U.S. Equal Employment Opportunity Commission (2010). Fact sheet on employment tests and selection procedures. Washington, DC: Author. Available...
  • Whitney, C. R., & Candelaria, C. A. (2017). The effects of No Child Left Behind on children’s socioemotional outcomes. AERA Open, 3(3),...
  • Zenisky, A. L., Sireci, S. G., Lewis, J., Lim, H., O’Donnell, F., Wells, C. S., Padellaro, F., Jung, H., Banda, E., Pham, D. Hong, S., Park,...