Change in the center of the distribution and in the individual scores: Relation with heteroskedastic pre- and post-test distributions

• Eduardo Estrada ^[1] ; José Manuel Caperos ^[2] ; Antonio Pardo ^[1]
  1. [1] Universidad Autónoma de Madrid

     Universidad Autónoma de Madrid

     Madrid, España

     
  2. [2] Universidad Pontificia Comillas

     Universidad Pontificia Comillas

     Madrid, España

     
• Localización: Psicothema, ISSN-e 1886-144X, ISSN 0214-9915, Vol. 32, Nº. 3, 2020, págs. 410-419
• Idioma: inglés
• DOI: 10.7334/psicothema2019.396
• Títulos paralelos:
  • Cambio en el centro de la distribución y en las puntuaciones individuales: : relación con distribuciones heteroscedasticas pre y post prueba
• Enlaces
  • Texto completo (pdf)
• Resumen
  • español

    Antecedentes: aunque el tamaño del efecto (ES) y el porcentaje de cambios individuales (PIC) son cosas distintas, no parecen ser independientes: mayores ESs conllevan mayores PICs. Pero esta relación todavía no ha sido suficientemente explorada. Método: estudiamos dicha relación mediante datos simulados en el contexto de un diseño pre-post con y sin grupo control. En la simulación se han utilizado diferentes distribuciones, tamaños muestrales, niveles de fiabilidad test-retest, efectos de varios tamaños y distintas variabilidades en el pre- y en el post-test. Resultados: el PIC está estrechamente relacionado con el ES. En los diseños pre-post, el modelo de regresión lineal ofrece valores R2 por encima de 0,90. En los diseños pre-post con grupo control, valores R2 por encima de 0,80. Estos resultados se mantienen incluso cuando la variabilidad del post-test es distinta de la del pre-test. Conclusiones: (1) cuando no se tiene información sobre el PIC, el tamaño del efecto puede utilizarse para estimar ese porcentaje; (2) el PIC sirve para precisar el significado de las medidas del tamaño del efecto.

  • English

    Background: Although average-based effect size (ES) and percentage of individual changes (PIC) are quite different, they are not independent: larger ESs lead to higher PICs. However, this association has not been sufficiently explored. Method: We analyzed this association based on data simulated in the context of a pre-post design, with and without control groups. We simulated various distributions, sample sizes, degrees of test-retest reliability, effect sizes, and different variances in pre- and post-test. Results: The PIC is closely associated with the ES across a wide variety of empirically frequent scenarios. In the “single group pre-post designs”, the linear regression model shows R2 values above 0.90. In the “control group pre-post designs”, the linear regression model shows R2 values above 0.80. These results were found even when the post-test variability differed from that of the pre-test, replicating, extending and generalizing the findings in previous studies. Conclusions: (1) In the absence of information about the PIC, the ES may be used to estimate this percentage. (2) The PIC is useful in interpreting the meaning of ES measures.

• Referencias bibliográficas
  • Beaton, D. E., Bombardier, C., Katz, J. N., & Wright, J. G. (2001). A taxonomy for responsiveness. Journal of Clinical...
  • Bentler, P. M., & Mooijaart, A. (1989). Choice of structural model via parsimony: A rationale based on precision. Psychological...
  • Blanca, M. J., Arnau, J., López-Montiel, D., Bono, R., & Bendayan, R. (2013). Skewness and kurtosis in real data samples....
  • Cecchini, J. A., González, C., Llamedo, R., Sánchez, B., & Rodríguez, C. (2019). The impact of cooperative learning ...
  • Christensen, L., & Mendoza, J. L. (1986). A method of assessing change in a single subject: An alteration of the RC index. Behavior...
  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed). Hillsdale, NJ: L. Erlbaum Associates.
  • Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155-159. https://doi.org/10.1037/0033-2909.112.1.155
  • Crawford, J. R., & Howell, D. C. (1998). Regression equations in clinical neuropsychology: An evaluation of statistical methods ...
  • Crosby, R. D., Kolotkin, R. L., & Williams, G. R. (2003). Defining clinically meaningful change in health-related quality of...
  • de Vet, H. C., Terwee, C. B., Ostelo, R. W., Beckerman, H., Knol, D. L., & Bouter, L. M. (2006). Minimal changes in health status questionnaires:...
  • Estrada, E., Ferrer, E., & Pardo, A. (2019). Statistics for evaluating pre-post change: Relation between change in the distribution center...
  • Ferrer, R., & Pardo, A. (2014). Clinically meaningful change: False positives in the estimation of individual change....
  • Foster, J. L., Harrison, T. L., Hicks, K. L., Draheim, C., Redick, T. S., & Engle, R. W. (2017). Do the effects ...
  • Grissom, R. J., & Kim, J. J. (2012). Effect sizes for research: Univariate and multivariate applications (2nd ed). New York: Routledge.
  • Hays, W. L. (1988). Statistics (2a ed.). Chicago, IL.: Holt, Rinehart and Winston.
  • Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy...
  • Kirk, R. E. (2013). Experimental design: Procedures for the behavioral sciences (4th ed). Thousand Oaks: Sage Publications.
  • Lemieux, J., Beaton, D. E., Hogg-Johnson, S., Bordeleau, L. J., & Goodwin, P. J. (2007). Three methods for minimally important ...
  • Maassen, G. H. (2004). The standard error in the Jacobson and Truax Reliable Change Index: The classical approach to the...
  • Macías, J., Valero-Aguayo, L., Bond, F. W., & Blanca, M. J. (2019). The efficacy of functional-analytic psychotherapy and acceptance...
  • Norman, G. R., Sridhar, F. G., Guyatt, G. H., & Walter, S. D. (2001). Relation of distribution and anchor-based approaches...
  • Nunnally, J. C., & Bernstein, I. H. (1994).Psychometric theory (3rd ed.).New York: McGraw-Hill.
  • Ogles, B. M., Lunnen, K. M., & Bonesteel, K. (2001). Clinical significance: History, application, and current practice. Clinical Psychology...
  • Osma, J., Sánchez-Gómez, A., & Peris-Baquero, Ó. (2018). Applying the unified protocol to a single case of major depression with schizoid...
  • Pardo, A., & Ferrer, R. (2013). Clinical significance: False positives in the estimation of individual change. Anales de Psicología, 29(2),...
  • Payne, R. W., & Jones, H. G. (1957). Statistics for the investigation of individual cases. Journal of Clinical Psychology, 13(2),...
  • Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, N.J:...
  • Pek, J., & Flora, D. B. (2018). Reporting effect sizes in original psychological research: A discussion and tutorial. Psychological Methods,...
  • Schmitt, J. S., & Di Fabio, R. P. (2004). Reliable change and minimum important difference (MID) proportions facilitated...
  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference....
  • Smole , T., Jastrzebski, J., Estrada, E., & Chuderski, A. (2018). Most evidence for the compensation account of cognitive training is...
  • Steele, A. G., & Douglas, R. J. (2006). Simplicity with advanced mathematical tools for metrology and testing....
  • Vindras, P., Desmurget, M., & Baraduc, P. (2012). When one size does not fi t all: A simple statistical method to deal...