Performance evaluation of recent information criteria for selecting multilevel models in Behavioral and Social Sciences

• Guillermo Vallejo Seco ^[1] ; Ellián Tuero Herrero ^[1] ; José Carlos Núñez Pérez ^[1] ; Pedro Rosário ^[2]
  1. [1] Universidad de Oviedo

     Universidad de Oviedo

     Oviedo, España

     
  2. [2] Universidade do Minho

     Universidade do Minho

     Braga (São José de São Lázaro), Portugal

     
• Localización: International journal of clinical and health psychology, ISSN 1697-2600, Vol. 14, Nº. 1, 2014, págs. 48-57
• Idioma: inglés
• DOI: 10.1016/s1697-2600(14)70036-5
• Enlaces
  • Texto completo (pdf)
• Dialnet Métricas: 6 Citas
• Resumen

  • This study was designed to find the best strategy for selecting the correct multilevel model among several alternatives taking into account variables such as intraclass correlation, number of groups (m), group size (n), or others as parameter values and intercept-slope covariance. First, we examine this question in a simulation study and second, to illustrate the behavior of the criteria and to explore the generalizability of the findings, a previously published educational dataset is analyzed. The results showed that none of the selection criteria behaved correctly under all the conditions or was consistently better than the others. The intraclass correlation somewhat affects the performance of all selection criteria, but the extent of this influence is relatively minor compared to sample size, parameter values, and correlation between random effects. A large number of groups appears more important than a large number of individuals per group in selecting the best model (m = 50 and n = 20 is suggested). Finally, model selection tools such as Akaike's Information Criterion (AIC) or the conditional AIC are recommend when it is assumed that random effects are correlated, whereas use of the Schwarz's Bayesian Information Criterion or the consistent AIC are advantageous for uncorrelated random effects

• Referencias bibliográficas
  • Best, N., Mason, A., Richardson, S.(2012).Two-pronged strategy for using DIC to compare selection models with non-ignorable missing responses.Bayesian...
  • Burnham, K.P., Anderson, D.R., Huyvaert, K.P.(2011). AIC model selection and multimodel inference in behavioral ecology: Some background,...
  • Dettmers, S., Trautwein, U., Lüdtke, O., Kunter, M., & Baumert, J.(2010).Homework works if homework quality is high: Using multilevel...
  • Gibbons, R.D., Hedeker, D., & DuToit, S.(2010).Advances in analysis of longitudinal data.Annual Review of Clinical Psychology, 6, 79-107.
  • Greven, S., & Kneib, T.(2010).On the behaviour of marginal and conditional AIC in linear mixed models.Biometrika, 97, 1-17.
  • Gurka, M.J.(2006).Selecting the best linear mixed model under REML.The American Statistician, 60, 19-26.
  • Hamaker, E.L., Van Hattum, P., Kuiper, R.M., & Hoijtink, H.(2011).Model selection based on information criteria in multilevel modeling.In...
  • Hartley, J.(2012).New ways of making academic articles easier to read.International Journal of Clinical and Health Psychology,12, 143-160.
  • Hox, J.J.(2010).Multilevel analysis.Techniques and applications (2th.ed.).New York: Routledge.
  • Imel, Z.E., Hubbard, R.A., Rutter, C.M., & Simon, G.(2013).Patient-rated alliance as a measure of therapist performance in two clinical...
  • Johnson, J.B., & Omland, K.S.(2004).Model selection in ecology and evolution.Trends in Ecology and Evolution, 19, 101-108.
  • Lee, H., & Ghosh, S.K.(2009).Performance of information criteria for spatial models.Journal of Statistical Computation and Simulation,...
  • Maas, C.M., & Hox, J.J.(2004).The influence of violations of assumptions on multilevel parameter estimates and their standard errors.Computational...
  • Núñez, J.C., Rosário, P., Vallejo, G., & González-Pienda, J.A.(2013).A longitudinal assessment of the effectiveness of a school-based...
  • Núñez, J.C., Vallejo, G., Rosário, P., Tuero-Herrero, E., & Valle,A.(in press).Variables from the students, the teachers and the school...
  • Snijders, T., & Bosker, R.(2012).Multilevel analysis: An introduction to basic and advanced multilevel modeling (2th.ed.).Thousand Oaks,...
  • Sobral, J., Villar, P., Gómez-Fraguela, J.A., Romero, E., & Luengo,M.A.(2013).Interactive effects of personality and separationas acculturation...
  • Spiegelhalter, D.J., Best, N.G., Carlin, B.P., & Van der Linde, A.(2002).Bayesian measures of model complexity and fit.Journal of the...
  • Srivastava M.S., & Kubokawa, T.(2010).Conditional information criteria for selecting variables in linear mixed models.Journal of Multivariate...
  • Sterba, S.K., & Pek, J.(2012).Individual influence on model selection.Psychological Methods, 17, 582-599.
  • Vaida, F., & Blanchard, S.(2005).Conditional Akaike information for mixed-effects models.Biometrika, 92, 351-370.
  • Verbeke, G., & Molenberghs, G.(2009).Linear mixed models for longitudinal data (3th.ed.).New York: Springer-Verlag.
  • Vallejo, G., Arnau, J., Bono, R., Fernández, M.P., & Tuero-Herrero,E.(2010).Nested model selection for longitudinal data using information...
  • Vallejo, G., Ato, M., & Valdés, T.(2008).Consequences of misspecifying the error covariance structure in linear mixed models for longitudinal...
  • Vallejo, G., Fernández, M.P., Livacic-Rojas, P.E., & Tuero-Herrero, E.(2011).Selecting the best unbalanced repeated measures model.Behavior...
  • Wang, J., & Schaalje, G. B. (2009). Model selection for linear mixed models using predictive criteria. Communications in Statistics -...