The hard but necessary task of gathering order-one effect size indices in meta-analysis

• Autores: María del Carmen Ortego Maté, Juan Botella Ausina
• Localización: Psicológica: Revista de metodología y psicología experimental, ISSN-e 1576-8597, ISSN 0211-2159, Vol. 31, Nº 2, 2010, págs. 291-315
• Idioma: inglés
• Enlaces
  • Texto completo (pdf)
• Dialnet Métricas: 1 Cita
• Resumen

  • Meta-analysis of studies with two groups and two measurement occasions must employ order-one effect size indices to represent study outcomes. Especially with non-random assignment, non-equivalent control group designs, a statistical analysis restricted to post-treatment scores can lead to severely biased conclusions. The 109 primary studies included in 4 metaanalyses were recovered, and their authors were contacted to request the raw data to calculate the order-one effect size indices. From this total we only got 13 primary studies. The results with the raw data analysis were compared with those performed with the order-zero and order-one indices. Despite the difficulties for gathering the data, the few data sets analyzed show that if the meta-analysis is performed with order-zero indices, the results can be severely misleading.

• Referencias bibliográficas
  • American Medical Association (1994) Current opinions of the medical council on ethical and judicial affairs. E-9.08 new medical procedures....
  • American Psychological Association (2006). Publication Manual of the American Psychological Association (5th ed.). Washington, DC: American...
  • Baranowski, T., Baranwski, J. C., Cullen, K. W., Thompson, D. I., Nicklas, T., Zakeri I. F. & Rochon, J. (2003). The fun, food, and fitness...
  • Bayne-Smith, M., Fardy, P. S., Azzollini, M. S., Magel, J., Schmitz K. H. & Agin, D. (2004). Improvements in heart health behaviors and...
  • Becker, B. J. (1988). Synthesizing standardized mean-change measures. British Journal of Mathematical & Statistical Psychology, 41, 257-278
  • Bormann, J. E., Gifford, A. L., Shively, M., Smith, T. L., Redwine, L., Kelly, A., Becker, S., Gershwin, T. L., Bone, P. & Belding, W....
  • Botella, J. & Gambara, H. (2002). Qué es el meta-análisis. Madrid: Biblioteca Nueva.
  • Botella, J. & Gambara, H. (2006). Doing and reporting a meta-analysis. International Journal of Clinical and Health Psychology, 6(2),...
  • Botella, J. & Ortego, C. (in press). Compartir datos: hacia una investigación más sostenible. Psicothema.
  • Burke, V., Milligan, R., Thompson, C., Taggart, A., Dunbar, D., Spencer, M., Medland, A., Gracey, M., Vandongen, R. & Beilin, L. (1998)....
  • Christensen, R., Kristensen, P. K, Bartels, E. M, Bliddal, H. & Astrup, A. (2007). Efficacy and safety of weight-loss drug rimonabant:...
  • Cohen, J. (1977). Statistical power analysis for the behavioral sciences (2a ed.). Hillsdale, NJ: Lawrence Erlbaum.
  • Cohen, J. (1994). The Earth is round (p<.05). American Psychologist, 49, 997-1003
  • Cook, T. D. & Campbell, D. T. (1979). Quasi-experimentation: design and analysis issues for field setting. Rand McNally: Chicago.
  • Cooper, H. M. (1989). Integrating research: a guide for literature reviews (2nd ed.). Newbury Park, CA: Sage.
  • Dunlap W. P., Cortina J. M., Vaslow J. B., & Burke M. J. (1996). Meta-Analysis of experiments with matched groups or repeated measures....
  • Ettelson, R. G. (2002). The treatment of adolescent depression. Unpublished doctoral dissertation. Illinois State University.
  • Fisher, R. A, (1928). The general sampling distribution of the multiple correlation coefficient. Proceeding of Royal Society of London, Series...
  • Fleiss, J. L. (1994). Measures of effect size for categorical data. In H. Cooper, and L.V. Hedges (Eds.), The Handbook of Research Synthesis...
  • Gibbons, R. D., Hedeker, D. & Davis, J. M. (1993). Estimation of effect size from a series of experiments involving paired comparisons....
  • Glaser, R. R (2002). Accuracy of effect size calculation methods for repeated measures.and A8COVA data. Unpublished doctoral dissertation....
  • Glass, G. (1976). Primary, secondary and meta-analysis of research. Educational Research, 5, 3-8.
  • Hedges, L. V. (1981). Distribution theory for Glass's estimador of effect size and related estimators. Journal of Statistics, 6(2), 107-126.
  • Hedges, L. V. & Olkin, I. (1985). Statistical methods for meta-anlysis. Orlando: Academic Press.
  • Hedges, L. V. & Vevea, J. L. (1998). Fixed- and random-effects models in meta analysis. Psychological Methods, 3, 486-504.
  • Hopper, C. A, Munoz, K. D., Gruber, M. D., MacConnie, S., Schonfeldt, B. & Shunk, T. (1996). A school-based cardiovascular exercise and...
  • Huberty, C. J. (2002). A history of effect size indices. Educational and Psychological Measurement, 62, 227-240.
  • Huedo, T. (2006). Índices de tamaño del efecto para meta-analizar diseños pretest-postest con dos grupos: Un estudio de simulación Monte Carlo....
  • Johnson B., Mullen B. & Salas E. (1995). Comparisons of three major meta-analytic approaches. Journal of Applied Psychology 80, 94-106
  • Kain, J., Uauy, R., Albala, Vio F., Cerda R. and Leyton B. (2004). School-based obesity prevention in Chilean primary school children: methodology...
  • Keef, S. P. & Roberts, L. A. (2004). The meta-analysis of partial effect sizes. British Journal of Mathematical and Psychology, 57, 97-129.
  • Keppel G. & Wickens, T. D. (2004). Design and analysis: a researcher's handbook (4th ed.). Englewood Cliffs, NJ: Prentice-Hall.
  • Kirk, R. E. (1995). Experimental design. Procedures for the behavioural sciences (3rd ed.). Belmont, CA: Brooks/Cole.
  • Lipsey, M. W. & Wilson, D. B. (2001). Practical Meta-analysis. Thousand Oaks, CA: Sage.
  • Marín F. & Sánchez, J. (1996). Estimadores del tamaño del efecto en meta-análisis: Un estudio Monte Carlo del sesgo y la eficiencia. Psicológica,...
  • Maxwell, S. E. & Delaney, H. D. (1990). Designing experiments and analyzing data: a model comparison perspective. Belmont, CA: Wadsworth.
  • McMurray, R. G., Harrell J. S., Bangdiwala, S. I., Bradley, C. B., Deng, S. & Levine, A. (2002) A school-based intervention can reduce...
  • Miller G. M. & Chapman J. P. (2001). Misunderstanding analysis of covariance. Journal of Abnormal Psychology, 110, 40-48.
  • Morris, S. B. (2000). Distribution of the standardized mean change effect size for meta-analysis on repeated measures. British Journal of...
  • Morris, S.B. (2008). Estimating effect sizes from pretest-posttest-control group designs. Organizational Research Methods, 11(2), 364-386.
  • Morris, S. B. & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs....
  • Nature (2006). Editorial. A fair share. 444. 653-654. Retrieved February 28, 2008, from: http://.nature.com/nature/journal/v444/n7120full/444653b.html
  • Neidig, J. L., Smith, B. A. & Brashers, D. E. (2003). Aerobic exercise training for depressive symptom management in adults living with...
  • Neumark-Sztainer, D., Story, M., Hannan, P. J., Stat, M. & Rex, J. (2003). New Moves: a school-based obesity prevention program for adolescent...
  • Orwin, R. G. (1994). Evaluating coding decisions. In: Cooper y Hedges (eds.). The handbook of research synthesis. (pp.139-162). The Rusell...
  • Peterson, R. A. & Brown, S. P. (2005). On the use of beta coefficients in meta-analysis. Journal of Applied Psychology, 90(1), 175-181.
  • Rohde, P., Clarke, G. N., Mace, D. E., Jorgensen, J. S & Seeley, J. (2004). An efficacy/effectiveness study of cognitive-behavioural treatment...
  • Rosenthal, R. (1991). Meta-analytic procedures for social research (rev. ed.). Newbury Park, CA: Sage.
  • Rosenthal, R. (1994). Parametric measures of effect size. In H. Cooper & L. V. Hedges (eds), The Handbook of Research Synthesis (pp. 231-244)....
  • Rosenthal, R. (1995). Writing meta-analytic reviews. Psychological Bulletin, 18, 183-192.
  • Sánchez-Meca, J., Marín-Martínez, F. & Chacón-Moscoso, S. (2003). Effect-size índices for dichotomized outcomes in meta-analysis. Psychological...
  • Scott-Sheldon, L. A. J., Kalichman S. C, Carey, M. P. & Fielder, R. L. (2008). Stress management interventions for HIV+ adults: A...
  • Shadish W. R. & Haddock C. K. (1994). Combining estimates of effect size. In: H. Cooper y L. V. Hedges (Eds.). The Handbook of Research...
  • Sikkema, K. J., Hansen, N. B, Kochman, A., Tate, D. C, & DiFranceisco, W. (2004). Outcomes from a randomized controlled trial of a group...
  • Stice, E., Shaw, H. & Marti, C. N. (2006). A meta-analytic review of obesity prevention programs for children and adolescents: the skinny...
  • Thompson, B. (2006). Research synthesis: Effect sizes. In: J.Green, G. Camilla y PB Elmore (eds). Handbook of complementary methods in education...
  • Van Breukelen, G. J. P. (2006). ANCOVA versus change from baseline had more power in randomized studies and more bias in nonrandomized studies....
  • Viechtbauer, W (2007). Approximating confidence intervals for standardized effect sizes in two-independent and two-dependent samples designs....
  • Weiss, J. J., Mulder, C. L., Antoni, M. H., de Vroome, E. M., Garssen, B., & Goodkin, K. (2003). Effects of a supportive-expressive group...
  • Weisz, J. R., McCarty, C. A. & Valeri, S. M. (2006). Effects of Psychotherapy for depression in children and adolescents: a meta-analysis....
  • Wicherts, J. M., Borsboom, D., Kats, J. & Molenaar, D. (2006). The poor availability of psychological research data for reanalysis. American...
  • WMA (2000). Word Medical Association Declaration of Helsinki. Ethical Principles for Medical Research involving Human Subjects. Edimburg:...