Power and sample size calculations for testing linear combinations of group means under variance heterogeneity with applications to meta and moderation analyses

• Gwowen Shieh ^[1] ; Show-Li Jan ^[2]
  1. [1] National Chiao Tung University

     National Chiao Tung University

     Taiwán

     
  2. [2] Chung Yuan Christian University

     Chung Yuan Christian University

     Taoyuan District, Taiwán

     
• Localización: Psicológica: Revista de metodología y psicología experimental, ISSN-e 1576-8597, ISSN 0211-2159, Vol. 36, Nº 2, 2015, págs. 367-390
• Idioma: inglés
• Enlaces
  • Texto completo (pdf)
• Dialnet Métricas: 1 Cita
• Resumen

  • The general formulation of a linear combination of population means permits a wide range of research questions to be tested within the context of ANOVA. However, it has been stressed in many research areas that the homogeneous variances assumption is frequently violated. To accommodate the heterogeneity of variance structure, the Welch–Satterthwaite procedure is commonly used as an alternative to the t test for detecting the substantive significance of a linear combination of mean effects. This article presents two approaches to power and sample size calculations for the Welch– Satterthwaite test. The usefulness and diversity of the suggested techniques are illustrated with two of the potential applications in meta and moderation analyses. The numerical assessments showed that the proposed approaches outperform the existing methods on the accuracy of power calculations and sample size determinations for meta and moderation studies. Computer algorithms are also developed to implement the recommended procedures in actual research designs.

• Referencias bibliográficas
  • Aguinis, H. (2004). Regression analysis for categorical moderators. New York, NY: Guilford Press.
  • Aguinis, H., Beaty, J. C., Boik, R. J., & Pierce, C. A. (2005). Effect size and power in assessing moderating effects of categorical variables...
  • Aguinis, H., Boik, R. J., & Pierce, C. A. (2001). A generalized solution for approximating the power to detect effects of categorical...
  • Aguinis, H., & Pierce, C. A. (1998). Heterogeneity of error variance and the assessment of moderating effects of categorical variables:...
  • Aguinis, H., Pierce, C. A., Bosco, F. A., Dalton, D. R., & Dalton, C. M. (2011). Debunking myths and urban legends about meta-analysis....
  • Alexander, R. A., & DeShon, R. P. (1994). Effect of error variance heterogeneity on the power of tests for regression slope differences....
  • American Psychological Association Publications and Communications Board Working Group on Journal Article Reporting Standards. (2008). Reporting...
  • Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic,...
  • Bond, C. F., Jr., Wiitala, W. L., & Richard, F. D. (2003). Meta-analysis of raw mean differences. Psychological Methods, 8, 406-418.
  • Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences...
  • Frazier, P. A., Tix, A. P., & Barron, K. E. (2004). Testing moderator and mediator effects in counseling psychology research. Journal...
  • Grissom, R. J. (2000). Heterogeneity of variance in clinical data. Journal of Consulting and Clinical Psychology, 68, 155-165.
  • Guo, J. H., & Luh, W. M. (2009). On sample size calculation for 2 × 2 fixed-effect ANOVA when variances are unknown and possibly unequal....
  • Hartung, J., Argac, D., & Makambi, K. H. (2002). Small sample properties of tests on homogeneity in one-way ANOVA and meta-analysis. Statistical...
  • Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic Press.
  • Hedges, L. V., & Pigott, T. D. (2001). The power of statistical tests in meta-analysis. Psychological Methods, 6, 203-217.
  • Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd.). New York, NY:...
  • Jan, S. L., & Shieh, G. (2011). Optimal sample sizes for Welch’s test under various allocation and cost considerations. Behavior Research...
  • Johnson, N. L., Kotz, S., & Balakrishnan, N. (1995). Continuous univariate distributions (2nd ed., Vol. 2). New York, NY: Wiley.
  • Kim, S. H., & Cohen, A. S. (1998). On the Behrens-Fisher problem: A review. Journal of Educational and Behavioral Statistics, 23, 356-377.
  • Kutner, M. H., Nachtsheim, C. J., Neter, J., & Li, W. (2005). Applied linear statistical models (5th ed.). New York, NY: McGraw Hill.
  • Levy, K. J. (1978). A priori contrasts under conditions of variance heterogeneity. Journal of Experimental Education, 47, 42-45.
  • Maxwell, S. E., & Delaney, H. D. (2004). Designing experiments and analyzing data: A model comparison perspective (2nd ed.). Mahwah, NJ:...
  • McCullagh, P., & Nelder, J. (1989). Generalized linear models (2nd ed.). London: Chapman & Hall/CRC.
  • McCulloch, C. E., Searle, S. R., & Neuhaus, J. W. (2008). Generalized, linear, and mixed models (2nd ed.). New York: Wiley.
  • Muncer, S., Taylor, S., & Craigie, M. (2002). Power dressing and meta-analysis: Incorporating power analysis into meta-analysis. Journal...
  • Ruscio, J., & Roche, B. (2012). Variance heterogeneity in published psychological research: A review and a new index. Methodology, 8,...
  • SAS Institute (2011). SAS/IML User’s Guide, Version 9.2. Cary, NC: SAS Institute Inc.
  • Satterthwaite, F. E. (1946). An approximate distribution of estimate of variance components. Biometrics Bulletin, 2, 110-114.
  • Tomarken, A. J., & Serlin, R. C. (1986). Comparison of ANOVA alternatives under variance heterogeneity and specific noncentrality structures....
  • Valentine, J. C., Pigott, T. D., & Rothstein, H. R. (2010). How many studies do you need? A primer on statistical power for meta-analysis....
  • Vallejo, G., Ato, M., & Fernandez, M. P. (2010). A robust approach for analyzing unbalanced factorial designs with fixed levels. Behavior...
  • Wahlsten, D. (1991). Sample size to detect a planned contrast and a one degree-of-freedom interaction effect. Psychological Bulletin, 110,...
  • Welch, B. L. (1938). The significance of the difference between two means when the population variances are unequal. Biometrika, 29, 350-362.
  • Welch, B. L. (1947). The generalization of Students’ problem when several different population variances are involved. Biometrika, 34, 28-35.