Evaluaciones Multilingües: mejores Prácticas para su Desarrollo, Vinculación y Evaluación

Louise Badham; María Elena Oliveri; Stephen Sireci

Evaluaciones Multilingües: mejores Prácticas para su Desarrollo, Vinculación y Evaluación

Badham, Louise ^[3] ; Oliveri, María Elena ^[1] ; Sireci, Stephan G. ^[2]
1. [1] Purdue University
  
  Purdue University
  
  Township of Wabash, Estados Unidos
2. [2] University of Massachusetts System
  
  University of Massachusetts System
  
  City of Boston, Estados Unidos
3. [3] International Baccalaureate (United Kingdom)
Mostrar afiliaciones +
Localización: Psicothema, ISSN-e 1886-144X, ISSN 0214-9915, Vol. 37, Nº. 3, 2025, págs. 1-11
Idioma: español
DOI: 10.70478/psicothema.2025.37.19
Títulos paralelos:
- Navigating Multi-Language Assessments: best Practices for Test Development, Linking, and Evaluation
Enlaces
- Texto completo
Resumen
- español
  Antecedentes: El desarrollo de evaluaciones en varios idiomas es enormemente complejo y afecta a todas las etapas, desde el desarrollo de las pruebas hasta la puntuación y la evaluación de las puntuaciones. Se necesitan diferentes enfoques para examinar la comparabilidad y mejorar la validez de las evaluaciones interlingües. Método: Se presenta una revisión de la literatura y las prácticas relacionadas con los métodos utilizados en diferentes áreas de la evaluación interlingüística. Resultados: Se ha pasado de la traducción del idioma de origen al idioma de destino al desarrollo simultáneo de items en varios idiomas. Se utilizan métodos cuantitativos y cualitativos para vincular y evaluar las evaluaciones en varios idiomas y proporcionar pruebas de validez. Conclusiones: Este artículo proporciona a los profesionales una visión general y recomendaciones de la literatura relacionada con el desarrollo de pruebas, la vinculación y la validación de evaluaciones producidas en varios idiomas.
- English
  Background: Developing assessments in multiple languages is hugely complex, impacting every stage from test development to scoring, and evaluating scores. Different approaches are needed to examine comparability and enhance validity in cross-lingual assessments. Method: A review of literature and practices relating to different methods used in cross-lingual assessment is presented. Results: There has been a shift from source-to-target language translation to developing items in multiple languages simultaneously. Quantitative and qualitative methods are used to link and evaluate assessments across languages and provide validity evidence. Conclusions: This article provides practitioners with an overview and research-based recommendations relating to test development, linking, and validation of assessments produced in multiple languages.
Referencias bibliográficas
- Alatli, B. (2020). Cross-cultural measurement invariance of the items in the Science Literacy Test in the Programme for International Student...
- Alatli, B. (2022). An investigation of cross-cultural measurement invariance and item bias of PISA 2018 reading skills items. International...
- Allalouf, A., Hambleton, R. K., & Sireci, S. G. (1999). Identifying the causes of DIF in translated verbal items. Journal of Educational...
- Allalouf, A., Rapp, J., & Stoller, R. (2009). Which item types are better suited to the linking of verbal adapted tests? International...
- American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014)....
- Angoff, W. H., & Cook, L. L. (1988). Equating the scores of the Prueba de Aptitud Académica and the Scholastic Aptitude Test (Report No....
- Asparouhov, T., & Muthén, B. (2014). Multiple-group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal,...
- Badham, L., & Furlong, A. (2023). Summative assessments in a multilingual context: What comparative judgment reveals about comparability...
- Blanco, C. (2024). 2024 duolingo language report. Duolingo. https://blog. duolingo.com/2024-duolingo-language-report/
- Boldt, R. F. (1969). Concurrent validity of the PAA and SAT for bilingual Dade School County high school volunteers (College Entrance Examination...
- Brislin, R. W. (1970). Back-translation for cross-cultural research. Journal of Cross-cultural Psychology, 1(3), 185–216. https://doi.org/10.1177/135910457000100301
- Cascallar, A. S., & Dorans, N. J. (2005). Linking scores from tests of similar content given in different languages: An illustration of...
- CTB/McGraw-Hill (1988). Spanish assessment of basic education: Technical report. McGraw Hill.
- Davidov, E. (2011). Nationalism and constructive patriotism: A longitudinal test of comparability in 22 countries with the ISSP. International...
- Davis, S. L., Buckendahl, C. W., & Plake, B. S. (2008). When adaptation is not an option: An application of multilingual standard setting....
- Dept, S., Ferrari, A., & Halleux, B. (2017). Translation and cultural appropriateness of survey material in large-scale assessments. In...
- Dorans, N. J., & Middleton, K. (2012). Addressing the extreme assumptions of presumed linkings. Journal of Educational Measurement, 49(1),...
- Ebbs, D., & Koršňáková, P. (2016). Translation and translation verification for TIMSS 2015. In Martin, M. O., Mullis I. V. & Martin...
- Ebbs, D., Flicop, S., Hidalgo, M. M., & Netten, A. (2021). Systems and instrument verification in PIRLS 2021. In Methods and procedures:...
- El Masri, Y. H., Baird, J-A., & Graesser, A. (2016) Language effects in international testing: The case of PISA 2006 science items. Assessment...
- Ercikan, K. (2002). Disentangling sources of differential item functioning in multilanguage assessments. International Journal of Testing,...
- Ercikan, K., & Koh, K. (2005). Examining the construct comparability of the English and French versions of TIMSS. International Journal...
- Ercikan, K., & Lyons-Thomas, J. (2013). Adapting tests for use in other languages and cultures. In K.Geisinger, B. Bracken, J. Carlson,...
- Ercikan, K., & Por, H. (2020). Comparability in multilingual and multicultural assessment contexts. In A. I. Berman, E. H. Haertel, &...
- Fischer, R., & Fontaine, J. R. J. (2011). Methods for investigating structural equivalence. In D. Matsumoto and F. J. R. van de Vijver...
- Gierl, M. J., & Khaliq, S. N. (2001). Identifying sources of differential item and bundle functioning on translated achievement tests:...
- Gökçe, S., Berberoglu, G., Wells, C. S., & Sireci, S. G. (2021). Linguistic distance and translation differential item functioning on...
- Goodwin, S., Bilsky, L., Mulcaire, P., & Settles, B. (2023, 26–28 April). Machine learning applications to develop tests in multiple languages...
- Grisay, A. (2003). Translation procedures in OECD/PISA 2000 international assessment. Language Testing, 20(2), 225–240. https://doi.org/10.1191/0265532203lt254oa
- Grisay, A., Gonzalez, E., & Monseur, C. (2009). Equivalence of item difficulties across national versions of the PIRLS and PISA reading...
- Hambleton, R. K. (1994). Guidelines for adapting educational and psychological tests: A progress report. European Journal of Psychological...
- Hambleton, R. K. (2005). Issues, designs, and technical guidelines for adapting tests into multiple languages and cultures. In R.K. Hambleton,...
- Hambleton, R. K., & Zenisky, A. L. (2011). Translating and adapting tests for cross-cultural assessments. In D. Matsumoto, & F. J....
- Hao, J., von Davier, A. A., Yaneva, V., Lottridge, S., von Davier, M., & Harris, D. J. (2024). Transforming assessment: The impacts and...
- Hernández, A., Hidalgo, M. D., Hambleton, R. K., & Gómez-Benito, J. (2020). International test commission guidelines for test adaptation:...
- Hulin, C.L., Drasgow, F., & Komocar, J. (1982). Applications of item response theory to analysis of attitude scale translations. Journal...
- Hulin, C.L., & Mayer, L.J. (1986). Psychometric equivalence of a translation of the Job Descriptive Index into Hebrew. Journal of Applied...
- International Baccalaureate Organization (2018). Assessment principles and practices—Quality assessments in a digital age. International Baccalaureate...
- International Baccalaureate Organization (2024). The IB Diploma Programme and Career-Related Programme: May 2024 assessment session final...
- International Test Commission. (2017). ITC Guidelines for translating and adapting tests (2nd edition). International Test Commission. https://www.intestcom.org/files/guideline_test_adaptation_2ed.pdf
- International Test Commission and Association of Test Publishers (2022). Guidelines for technology-based assessments. International Test Commission...
- Kolen, M., & Brennan, R. (2004). Test equating, scaling, and linking: Methods and practices (2nd edition). Springer-Verlag.
- Koršňáková, P., Dept, S., & Ebbs, D. (2020). Translation: The preparation of national language versions of assessment instruments. In...
- Lord, F. M. (1980). Applications of item response theory to practical testing problems. Lawrence Erlbaum Publishers.
- Lu, Y., & Sireci, S. G. (2007). Validity issues in test speededness. Educational Measurement: Issues and Practice, 26(4), 29–37. https://...
- Martin, M. O., von Davier, M., & Mullis, I. V. (2020). Methods and procedures: TIMSS 2019 technical report. International Association...
- McGrane, J., Kayton, H., Double, K., Woore, R., & El Masri, Y. (2022). Is science lost in translation? Language effects in the International...
- Milman, L. H., Faroqi-Shah, Y., Corcoran, C. D., & Damele, D. M. (2018). Interpreting mini-mental state examination performance in highly...
- OECD. (2024). PISA 2022 Technical Report. OECD Publishing. https:// www.oecd.org/en/publications/pisa-2022-technical-report_01820d6d- en.html
- Oliveri, M. E., Olson, B., Ercikan, K., & Zumbo, B. D. (2012). Methodologies for investigating item- and test-level measurement equivalence...
- Oliveri, M. E., & von Davier, M. (2011). Investigation of model fit and score scale comparability in international assessments. Psychological...
- Oliveri, M. E., & von Davier, M. (2014). Toward increasing fairness in score scale calibrations employed in international large-scale...
- Oliveri, M. E., & von Davier, M. (2017). Analyzing invariance of item parameters used to estimate trends in international large-scale...
- Ong, S. L., & Sireci, S. G. (2008). Using bilingual students to link and evaluate different language versions of an exam. US-China Education...
- Rapp, J., & Allalouf, A. (2003). Evaluating cross-lingual equating. International Journal of Testing, 3(2), 101–117. https://doi.org/10.1207/...
- Robin, F., Sireci, S. G., & Hambleton, R. K. (2003). Evaluating the equivalence of different language versions of a credentialing exam....
- Rogers, W. T., Gierl, M. J., Tardif, C., Lin, J., & Rinaldi, C. M. (2003). Differential validity and utility of successive and simultaneous...
- Rogers, W. T., Gierl, M. J., Tardif, C., Lin, J., & Rinaldi, C. M. (2010). Validity of the simultaneous approach to the development of...
- Sireci, S. G. (2005). Using bilinguals to evaluate the comparability of different language versions of a test. In R.K. Hambleton, P. Merenda,...
- Sireci, S. G., & Oliveri, M. E. (2023). A Critical Review of the International Baccalaureate Organization’s Multilingual Assessment Processes...
- Sireci, S. G., Rios, J. A., & Powers, S. (2016). Comparing test scores from tests administered in different languages. In N. J. Dorans...
- Sireci, S. G., & Wells, C. S. (2010). Evaluating the comparability of English and Spanish video accommodations for English language learners....
- Sukin, T., Sireci, S. G., & Ong, S. L. (2015). Using bilingual examinees to evaluate the comparability of test structure across different...
- Tanzer, N. K. (2005). Developing tests for use in multiple languages and cultures: A plea for simultaneous development. In R. K. Hambleton,...
- van de Vijver, F. J. R., & Tanzer, N. K. (1997). Bias and equivalence in cross-cultural assessment: An overview. European Review of Applied...
- van de Vijver, F. J. R., & Poortinga, Y. H. (2005). Conceptual and methodological issues in adapting tests. In R. K. Hambleton, P. Merenda,...
- van de Vijver, F., Avvisati, F., Davidov, E., Eid, M., Fox, J. P., Le Donné, N., Lek, K., Meuleman, B., Paccagnella, M., & Van de Schoot,...
- Wainer, H., Sireci, S. G., & Thissen, D. (1991). Differential testlet functioning: Definitions and detection. Journal of Educational Measurement,...
- Wolff, H. G., Schneider-Rahm, C. I., & Forret, M. L. (2011). Adaptation of a German multidimensional networking scale into English. European...
- Woodcock, R. W., & Muñoz-Sandoval, A. F. (1993). An IRT approach to cross-language test equating and interpretation. European Journal...
- Zhao, X., Solano-Flores, G., & Qian, M. (2018). International test comparisons: Reviewing translation error in different source language-...
- Zumbo, B. D., Liu, Y., Wu, A. D., Shear, B. R., Astivia, O. L. O., & Ark, T. K. (2015). A methodology for Zumbo’s third generation DIF...