Convergent and Discriminant Validity Evidence of the Methodological Quality Scale for Studies Based on Observational Methodology (MQSOM)

• López-Arenas, Daniel ^[1] ; Sanduvete-Chaves, Susana ^[1] ; Mena-Raposo, José ^[1] ; Chacón-Moscoso, Salvador ^[1]
  1. [1] Universidad de Sevilla

     Universidad de Sevilla

     Sevilla, España

     
• Localización: The Spanish Journal of Psychology, ISSN 1138-7416, Nº 29, 2026
• Idioma: inglés
• DOI: 10.1017/SJP.2026.10021
• Enlaces
  • Texto completo
• Resumen

  • The use of observational methodology has become increasingly more common in psychological research, highlighting the need for tools that ensure methodological rigor. This study presents evidence of convergent/discriminant validity for the Methodological Quality Scale for Studies Based on Observational Methodology (MQSOM). A multitrait-multimethod (MTMM) analysis with Spearman’s correlations was used to examine the relationship between MQSOM dimensions and those of three instruments: the Methodological Rigor in Mixed Methods (MRMM), the Guidelines for Reporting Evaluations Based on Observational Methodology (GREOM), and the Mixed Methods Appraisal Tool (MMAT). Ninety-six articles were coded using MQSOM and the instruments for comparison. The MQSOM’s design converged with the MRMM’s mixed-methods design (ρ = .217, p = .034), GREOM’s design (ρ = .217, p = .034), and MMAT’s qualitative (QUAL) component (ρ = .212, p = .038). The MQSOM’s measurement and analysis aligned with MRMM’s data analysis (ρ = .611, p < .001), GREOM’s data quality control (ρ = .423, p < .001) and results (ρ = .328, p = .001), and MMAT’s quantitative (QUANT) (ρ = .214, p = .037) and mixed-methods (ρ = .643, p < .001) components. MQSOM’s design exhibited discriminant validity from MRMM’s data collection (ρ = .025, p = .807) and data analysis (ρ = −.051, p = .620), GREOM’s data quality control (ρ = .025, p = .812) and results (ρ = −.032, p = .759), and MMAT’s QUANT component (ρ = −.035, p = .733). This study reinforces the validity of MQSOM as a useful methodological quality scale.

• Referencias bibliográficas
  • Anguera, M. T., Blanco-Villaseñor, A., Losada, J. L., & Portell, M. (2018). Guidelines for designing and conducting a study that applies...
  • Anguera, M. T., Blanco-Villaseñor, A., Losada, J. L., & Sánchez-Algarra, P. (2020). Integración de elementos cualitativos y cuantitativos...
  • Anguera, M. T., Camerino, O., Castañer, M., Sánchez-Algarra, P., & Onwuegbuzie, A. (2017). The specificity of observational studies in...
  • Booth, A., Hannes, K., Harden, A., Noyes, J., Harris, J., & Tong, A. (2014). Chapter 21. COREQ (consolidated criteria for reporting qualitative...
  • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin,...
  • Chacón-Moscoso, S., Anguera, M. T., Sanduvete-Chaves, S., Losada, J. L., & Portell, M. (2019). Methodological quality checklist for studies...
  • Chacón-Moscoso, S., Sanduvete-Chaves, S., Portell, M., & Anguera, M. T. (2013). Reporting a program evaluation: Needs, program plan, intervention,...
  • Cochran, W. G., & Chambers, S. P. (1965). The planning of observational studies of human populations. Journal of the Royal Statistical...
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences. L. Erlbaum Associates.
  • Cresswell, J. W., & Plano-Clark, V. (2017). Designing and conducting mixed methods research. Sage.
  • Elliott, R., Fischer, C. T., & Rennie, D. L. (1999). Evolving guidelines for publication of qualitative research studies in psychology...
  • Harrison, R., Reilly, T., & Creswell, J. (2020). Methodological rigor in mixed methods: An application in management studies. Journal...
  • Holgado-Tello, F. P. (2015). Análisis de los ítems. [analysis of the items]. In Barbero-García, M. I., Vila-Abad, E., & Holgado-Tello,...
  • Hong, Q. N., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M. P., Griffiths, F., Nicolau, B., O’Cathain, A.,...
  • Hong, Q. N., Pluye, P., Fàbregues, S., Bartlett, G., Boardman, F., Cargo, M., Dagenais, P., Gagnon, M. P., Griffiths, F., Nicolau, B., O’Cathain,...
  • Kalkbrenner, M. T. (2023). Alpha, omega, and H internal consistency reliability estimates: Reviewing these options and when to use them. Counseling...
  • Leech, N., & Onwuegbuzie, A. (2010). Guidelines for conducting and reporting mixed research in the field of counseling and beyond. Journal...
  • Marsh, H. W., & Grayson, D. (1995). Latent variable models of multitrait-multimethod data. In Hoyle, R. H. (Ed.), Structural equation...
  • O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis...
  • O’Cathain, A., Murphy, E., & Nicholl, J. (2008). The quality of mixed methods studies in health services research. Journal of Health Services...
  • Portell, M., Anguera, M. T., Chacón-Moscoso, S., & Sanduvete-Chaves, S. (2015). Guidelines for reporting evaluations based on observational...
  • Portney, L., & Watkins, M. (2000). Foundations of clinical research: Applications to practice. Prentice Hall.
  • Sanduvete-Chaves, S., López-Arenas, D., Anguera, M. T., & Chacón-Moscoso, S. (2025). A scale for evaluating the methodological quality...
  • Stone, A. A., & Shiffman, S. (2002). Capturing momentary, self-report data: A proposal for reporting guidelines. Annals of Behavioral...
  • von Elm, E., Altman, D. G., Effer, M., Pocock, S. J., Gotzsche, P. C., & Vandenbroucke, J. P. (2007). The Strengthening the Reporting...