Testing measurement invariance of mathematics self-concept and self-efficacy in PISA using MGCFA and the alignment method

• Yi Ding ^[1] ; Kajsa Yang Hansen ^[1] ; Alli Klapp ^[1]
  1. [1] University of Gothenburg

     University of Gothenburg

     Suecia

     
• Localización: European Journal of Psychology of Education, ISSN-e 1878-5174, ISSN 0256-2928, Vol. 38, Nº 2, 2023, págs. 709-732
• Idioma: inglés
• DOI: 10.1007/s10212-022-00623-y
• Texto completo no disponible (Saber más ...)
• Dialnet Métricas: 2 Citas
• Resumen

  • The aim of the study is to investigate the measurement invariance of mathematics self-concept and self-efficacy across 40 countries that participated in the Programme for International Student Assessment (PISA) 2003 and 2012 cycles. The sample of the study consists of 271,760 students in PISA 2003 and 333,804 students in PISA 2012. Firstly, the traditional measurement invariance testing was applied in the multiple-group confirmatory factor analysis (MGCFA). Then, the alignment analyses were performed, allowing non-invariance to a minimum to estimate all of the parameters. Results from MGCFA indicate that mathematics self-concept and self-efficacy hold metric invariance across the 80 groups (cycle by country). The alignment method results suggest that a large proportion of non-invariance exists in both mathematics self-concept and self-efficacy factors, and the factor means cannot be compared across all participating countries. Results of the Monte Carlo simulation show that the alignment results are trustworthy. Implications and limitations are discussed, and some recommendations for future research are proposed.

• Referencias bibliográficas
  • Asparouhov, T., & Muthén, B. (2014). Multiple-group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal,...
  • Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Prentice-Hall.
  • Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman.
  • Beierlein, C., Davidov, E., Schmidt, P., Schwartz, S. H., & Rammstedt, B. (2012). Testing the discriminant validity of Schwartz’ Portrait...
  • Bollen, K. A. (1989). A new incremental fit index for general structural equation models. Sociological Methods & Research, 17(3), 303–316.
  • Bong, M., & Skaalvik, E. M. (2003). Academic self-concept and self-efficacy: How different are they really? Educational Psychology Review,...
  • Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford Publications.
  • Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling: A Multidisciplinary...
  • Chen, H., Dai, J., & Gao, Y. (2019). Measurement invariance and latent mean differences of the Chinese version physical activity self-efficacy...
  • Chen, P., & Zimmerman, B. (2007). A cross-national comparison study on the accuracy of self-efficacy beliefs of middle-school mathematics...
  • Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling,...
  • Chiu, M. (2017). Self-Concept, Self-Efficacy, and Mathematics Achievement: Students in 65 Regions Including the US and Asia (pp. 267–288)....
  • Cicero, D. C. (2020). Measurement invariance of the self-concept clarity scale across race and sex. Journal of Psychopathology and Behavioral...
  • Coromina, L., & Peral, E. B. (2020). Comparing alignment and multiple group CFA for analysing political trust in Europe during the crisis....
  • Davidov, E., Dülmer, H., Schlüter, E., Schmidt, P., & Meuleman, B. (2012). Using a multilevel structural equation modeling approach to...
  • Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., & Billiet, J. (2014). Measurement equivalence in cross-national research. Annual...
  • Eklöf, H. (2007). Self-concept and valuing of mathematics in TIMSS 2003: Scale structure and relation to performance in a Swedish setting....
  • Garcia, F., Martínez, I., Balluerka, N., Cruise, E., Garcia, O. F., & Serra, E. (2018). Validation of the five-factor self-concept questionnaire...
  • Githua, B. N., & Mwangi, J. G. (2003). Students’ mathematics self-concept and motivation to learn mathematics: Relationship and gender...
  • Glassow, L. N., Rolfe, V., & Yang Hansen, K. (2021). Assessing the comparability of teacher-related constructs in TIMSS 2015 across 46...
  • Harju-Luukkainen, H., Stang, J., & McElvany, N. (2020). Introduction to monitoring student achievement in the twenty-first century. In...
  • Harter, S. (1978). Effectance motivation reconsidered: Toward a developmental model. Human Development, 21(1), 34–64. https://doi.org/10.1159/000271574
  • He, J., Barrera-Pedemonte, F., & Buchholz, J. (2019). Cross-cultural comparability of noncognitive constructs in TIMSS and PISA. Assessment...
  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives....
  • Kim, E. S., Cao, C., Wang, Y., & Nguyen, D. T. (2017). Measurement invariance testing with many groups: A comparison of five approaches....
  • Lee, J. (2009). Universals and specifics of math self-concept, math self-efficacy, and math anxiety across 41 PISA 2003 participating countries....
  • Marsh, H. W. (1987). The big-fish-little-pond effect on academic self-concept. Journal of Educational Psychology, 79(3), 280.
  • Marsh, H. W., Abduljabbar, A. S., Morin, A. J. S., Parker, P., Abdelfattah, F., Nagengast, B., & Abu-Hilal, M. M. (2015). The big-fish-little-pond...
  • Marsh, H. W., Guo, J., Parker, P. D., Nagengast, B., Asparouhov, T., Muthén, B., & Dicke, T. (2018). What to do when scalar invariance...
  • Marsh, H. W., & Hau, K.-T. (2003). Big-fish-little-pond effect on academic self-concept: A cross-cultural (26-country) test of the negative...
  • Marsh, H. W., Hau, K.-T., Artelt, C., Baumert, J., & Peschar, J. L. (2006). OECD’s brief self-report measure of educational psychology’s...
  • Marsh, H. W., & Martin, A. J. (2011). Academic self-concept and academic achievement: Relations and causal ordering. British Journal of...
  • Marsh, H. W., & O’Mara, A. (2008). Reciprocal effects between academic self-concept, self-esteem, achievement, and attainment over seven...
  • Marsh, H. W., Pekrun, R., Parker, P. D., Murayama, K., Guo, J., Dicke, T., & Arens, A. K. (2019). The murky distinction between self-concept...
  • Marsh, H. W., Seaton, M., Trautwein, U., Lüdtke, O., Hau, K. T., O’Mara, A. J., & Craven, R. G. (2008). The big-fish–little-pond-effect...
  • Marsh, H. W., & Shavelson, R. (1985). Self-concept: Its multifaceted, hierarchical structure. Educational Psychologist, 20(3), 107–123....
  • McConney, A., & Perry, L. (2010). Socioeconomic status, self-efficacy, and mathematics achievement in Australia: A secondary analysis....
  • Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance....
  • Millsap, R. E. (2012). Statistical approaches to measurement invariance. Routledge.
  • Munck, I., Barber, C., & Torney-Purta, J. (2018). Measurement invariance in comparing attitudes toward immigrants among youth across Europe...
  • Muthén, B., & Asparouhov, T. (2013). New methods for the study of measurement invariance with many groups. Mplus www.statmodel.com.
  • Muthén, L. K., & Muthén, B. (1998). Mplus user’s guide: Statistical analysis with latent variables, user’s guide. Muthén & Muthén.
  • Niehaus, K., & Adelson, J. L. (2013). Self-concept and native language background: A study of measurement invariance and cross-group comparisons...
  • Odell, B., Gierl, M., & Cutumisu, M. (2021). Testing measurement invariance of PISA 2015 mathematics, science, and ICT scales using the...
  • OECD. (2013). PISA 2012 assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. Organisation...
  • OECD. (2014). PISA 2012 technical report. Organisation for Economic Co-Operation and Development. https://www.oecd.org/pisa/pisaproducts/PISA-2012-technical-report-final.pdf.
  • Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66(4), 543–578.
  • Parker, P. D., Marsh, H. W., Ciarrochi, J., Marshall, S., & Abduljabbar, A. S. (2014). Juxtaposing math self-efficacy and self-concept...
  • Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions...
  • Randhawa, B. S., Beamer, J. E., & Lundberg, I. (1993). Role of mathematics self-efficacy in the structural model of mathematics achievement....
  • Rogers, C. R. (1951). Client-centered therapy; its current practice, implications, and theory (pp. xii, 560). Houghton Mifflin.
  • Rosenberg, M. (1979). Conceiving the self. RE Krieger.
  • Rutkowski, L., & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the context of large-scale international surveys....
  • Schmitt, N., Golubovich, J., & Leong, F. T. (2011). Impact of measurement invariance on construct correlations, mean differences, and...
  • Schunk, D. H. (1989). Self-efficacy and achievement behaviors. Educational Psychology Review, 1(3), 173–208. https://doi.org/10.1007/BF01320134
  • Shavelson, R. J., Hubner, J. J., & Stanton, G. C. (1976). Self-concept: Validation of construct interpretations. Review of Educational...
  • Svetina, D., Rutkowski, L., & Rutkowski, D. (2020). Multiple-group invariance with categorical outcomes using updated guidelines: An illustration...
  • Teo, T., & Kam, C. (2014). A measurement invariance analysis of the general self-efficacy scale on two different cultures. Journal of...
  • Usher, E. L., & Pajares, F. (2006). Sources of academic and self-regulatory efficacy beliefs of entering middle school students. Contemporary...
  • Uysal, N. K., & Arıkan, Ç. A. (2018). Measurement invariance of science self-efficacy scale in PISA. 5(2), 14.
  • Van de Schoot, R., Lugtig, P., & Hox, J. (2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology,...
  • van de Vijver, F. J. (2018). Towards an integrated framework of bias in noncognitive assessment in international large-scale studies: Challenges...
  • Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and...
  • Widaman, K. F., & Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance...
  • Zhang, D., & Wang, C. (2020). The relationship between mathematics interest and mathematics achievement: Mediating roles of self-efficacy...