A comparison of discriminant logistic regression and Item Response Theory Likelihood-Ratio Tests for Differential Item Functioning (IRTLRDIF) in polytomous short tests

• María Dolores Hidalgo ^[1] ; María Dolores López-Martínez ^[1] ; Juana Gómez-Benito ^[2] ; Georgina Guilera ^[2]
  1. [1] Universidad de Murcia

     Universidad de Murcia

     Murcia, España

     
  2. [2] Universitat de Barcelona

     Universitat de Barcelona

     Barcelona, España

     
• Localización: Psicothema, ISSN-e 1886-144X, ISSN 0214-9915, Vol. 28, Nº. 1, 2016, págs. 83-88
• Idioma: inglés
• Títulos paralelos:
  • Funcionamiento diferencial del ítem en tests breves: comparación entre regresión logística discriminante e IRTLRDIF
• Enlaces
  • Texto completo (pdf)
• Dialnet Métricas: 2 Citas
• Resumen
  • español

    Antecedentes: en ciencias sociales, del comportamiento y de salud es habitual usar tests breves. El tamaño del test puede afectar a la correcta identificación de ítems con DIF. Este trabajo compara la eficacia relativa de la Regresión Logística Discriminante (RLD) e IRTLRDIF en la detección del DIF en tests cortos politómicos. Método: se diseñó un estudio de simulación. Se manipuló tamaño del test, tamaño de la muestra, cantidad DIF y número de categorías de respuesta al ítem. Se evaluó el Error Tipo I y la potencia.Resultados: en las condiciones de no-DIF IRTLRDIF y RLD mostraron tasas de Error Tipo I cercanas al nivel nominal. En tests con DIF las tasas de Error Tipo I dependieron del tamaño del test, de la muestra, cantidad de DIF, contaminación del test y número de categorías del ítem. RLD presentó mayor tasa de Error Tipo I que IRTLRDIF. La potencia estuvo afectada por la cantidad de DIF y tamaño de la muestra. En tests muy cortos RLD mostró mayor potencia que IRTLRDIF. Conclusiones: en tests cortos y con DIF las tasas de Error Tipo I fueron elevadas. La potencia de IRTLRDIF y RLD fue relativamente baja en tests cortos y tamaños muestrales pequeños.

  • English

    Background: Short scales are typically used in the social, behavioural and health sciences. This is relevant since test length can influence whether items showing DIF are correctly flagged. This paper compares the relative effectiveness of discriminant logistic regression (DLR) and IRTLRDIF for detecting DIF in polytomous short tests. Method: A simulation study was designed. Test length, sample size, DIF amount and item response categories number were manipulated. Type I error and power were evaluated. Results: IRTLRDIF and DLR yielded Type I error rates close to nominal level in no-DIF conditions. Under DIF conditions, Type I error rates were affected by test length DIF amount, degree of test contamination, sample size and number of item response categories. DLR showed a higher Type I error rate than did IRTLRDIF. Power rates were affected by DIF amount and sample size, but not by test length. DLR achieved higher power rates than did IRTLRDIF in very short tests, although the high Type I error rate involved means that this result cannot be taken into account. Conclusions: Test length had an important impact on the Type I error rate. IRTLRDIF and DLR showed a low power rate in short tests and with small sample sizes.

• Referencias bibliográficas
  • Balluerka, N., Gorostiaga, A., Gómez-Benito, J., & Hidalgo, M. D. (2010). Use of multilevel logistic regression to identify the causes...
  • Benítez, I., Padilla, J. L., Hidalgo, M. D., & Sireci, S. G. (in press). Using mixed methods to interpret differential item functioning....
  • Bolt, D. M. (2002). A Monte-Carlo comparison of parametric and nonparametric polytomous DIF detection methods. Applied Psychological Measurement,...
  • Bradley, J. V. (1978) Robustness? British Journal of Mathematical and Statistical Psychology, 31, 144-152.
  • Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Newbury Park, CA: Sage.
  • DeMars, C. E. (2008). Polytomous differential item functioning and violations of ordering of the expected latent trait by the Raw Score. Educational...
  • Donoghue, J. R., Holland, P. W., & Thayer, D. T. (1993). A Monte Carlo study of factors that affect the Mantel-Haenszel and standardization...
  • Finch, W. H., & French, B. F. (2008). Anomalous Type I Error rates for identifying one type of differential item functioning in the presence...
  • French, A. W., & Miller, T. R. (1996). Logistic regression and its use in detecting differential item functioning in polytomous items....
  • Gelin, M. N., & Zumbo, B. D. (2007). Operating characteristics of the DIF MIMIC approach using Jöreskog’s covariance matrix with ML and...
  • Gómez-Benito, J., Hidalgo, M. D., & Zumbo, B. (2013). Effi ciency of combination of R-square and Odds-ratio using discriminant logistic...
  • González-Betanzos, F., & Abad, F. J. (2012). The effects of purifi cation and the evaluation of Differential Item Functioning with the...
  • Guilera, G., Gómez-Benito, J., Hidalgo, M. D., & Sánchez-Meca, J. (2013). Type I Error and statistical power of Mantel-Haenszel procedure...
  • Hidalgo, M. D., Gómez-Benito, J., & Padilla, J. L. (2005). Regresión logística: alternativas de análisis en la detección del funcionamiento...
  • Miller, T. R., & Spray, J. A. (1993). Logistic discriminant function analysis for DIF identifi cation of polytomously scored items. Journal...
  • Paek, I., & Wilson, M. (2011). Formulating the Rasch Differential Item functioning model under the marginal maximum Likelihood estimation...
  • Penfield, R. D. (2007). An approach for categorizing DIF in polytomous items. Applied Measurement in Education, 20, 335-355.
  • Penfield, R. D., Alvarez, K., & Lee, O. (2009). Using a taxonomy of Differential Step Functioning to improve the interpretation of DIF...
  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph Supplement, 17.
  • Scott, N. W., Fayer, P. M., Aaronson, N. K., Bottomley, A., Graeff, A., Groenvold, M., …, Sprangers, M. A. G. (2009). A simulation study provided...
  • Scott, N. W., Fayers, P. M., Aaronson, N. K., Bottomley, A., Graeff, A., Groenvold, M., …, Sprangers, M. A. G. (2010). Differential Item Functioning...
  • Sireci, S. G., & Ríos, J. A. (2013). Decisions that make a difference in detecting differential item functioning. Educational Research...
  • Teresi, J. A. (2006). Overview of quantitative measurement methods: Equivalence, invariance, and differential item functioning in health applications....
  • Thissen, D. (2001). IRTLRDIF v.2.0b: Software for the computation of the statistics involved in item response theory likelihood-ratio tests...
  • Thissen, D., Steinberg, L., & Wainer, H. (1988). Use of item response theory in the study of group differences in trace lines. In Wainer,...
  • Wang, W. C., & Yeh, Y. L. (2003). Effects of anchor item methods on differential item functioning detection with the likelihood ratio...