Using the R-MAPE index as a resistant measure of forecast accuracy

• Autores: Juan José Montaño Moreno, Alfonso Luis Palmer Pol, Albert Sesé Abad, Berta Cajal Blasco
• Localización: Psicothema, ISSN-e 1886-144X, ISSN 0214-9915, Vol. 25, Nº. 4, 2013, págs. 500-506
• Idioma: inglés
• DOI: 10.7334/psicothema2013.23
• Títulos paralelos:
  • El índice R-MAPE como medida resistente del ajuste en la previsión
• Enlaces
  • Texto completo (pdf)
• Dialnet Métricas: 1 Cita
• Resumen
  • español

    Antecedentes: el Promedio del Error Porcentual Absoluto (MAPE) es probablemente la medida de adecuación de la previsión más ampliamente utilizada. Sin embargo, no cumple el criterio de validez debido a que la distribución de los errores porcentuales absolutos habitualmente presenta una forma asimétrica a la derecha con presencia de valores alejados. En estos casos, el MAPE proporciona una sobreestimación del correspondiente parámetro poblacional. En el presente trabajo se propone un índice alternativo, denominado MAPE Resistente o R-MAPE, y basado en el cálculo del M-estimador de Huber, el cual permite superar la mencionada limitación. Método: se utilizan los resultados derivados de la aplicación de modelos de Red Neuronal Artificial (ANN) y modelos Autorregresivos Integrados de Media Móvil (ARIMA) en la previsión de una serie temporal. Resultados: se puede observar que la media aritmética, el MAPE, realiza una sobreestimación del correspondiente parámetro poblacional, a diferencia del R-MAPE, sobre un conjunto de distribuciones de errores con asimetría a la derecha y presencia de valores alejados. Conclusiones: nuestros resultados ponen de manifiesto que el R-MAPE representa una adecuada alternativa en la medición del ajuste en la previsión, debido a que proporciona una evaluación válida de dicho ajuste en comparación al MAPE.

  • English

    Background: The mean absolute percentage error (MAPE) is probably the most widely used goodness-of-fit measure. However, it does not meet the validity criterion due to the fact that the distribution of the absolute percentage errors is usually skewed to the right, with the presence of outlier values. In these cases, MAPE overstates the corresponding population parameter. In this study, we propose an alternative index, called Resistant MAPE or R-MAPE based on the calculation of the Huber M-estimator, which allows overcoming the aforementioned limitation. Method: The results derived from the application of Artificial Neural Network (ANN) and Autoregressive Integrated Moving Average (ARIMA) models are used to forecast a time series. Results: The arithmetic mean, MAPE, overstates the corresponding population parameter, unlike R-MAPE, on a set of error distributions with a statistically significant right skew, as well as outlier values. Conclusions: Our results suggest that R-MAPE represents a suitable alternative measure of forecast accuracy, due to the fact that it provides a valid assessment of forecast accuracy compared to MAPE.

• Referencias bibliográficas
  • Ahlburg, D. (1995). Simple versus complex models: Evaluation, accuracy, and combining. Mathematical Population Studies, 5, 281-290.
  • Armstrong, J.S., & Collopy, F. (1992). Error measures for generalizing about forecasting methods: Empirical comparisons. International...
  • Balestrino, A., Bini Verona, F., & Santanche, M. (1994). Time series analysis by neural networks: Environmental temperature forecasting....
  • Box, G., & Cox, D. (1964). An analysis of transformations. Journal of the Royal Statistical Society, Series B, 26, 211-252.
  • Box, G., & Jenkins, G.M. (1976). Time series analysis: Forecasting and control. San Francisco: Holden Day.
  • Carmines, E., & Zeller, R. (1979). Reliability and validity assessment. Beverly Hills, CA: Sage.
  • Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Mathematical Control, Signal and Systems, 2, 303-314.
  • Escudero, J.R., & Vallejo, G. (2000). Aplicación del diseño de series temporales múltiples a un caso de intervención en dos clases de...
  • Fildes, R. (1992). The evaluation of extrapolative forecasting methods. International Journal of Forecasting, 8, 81-98.
  • Foster, W.R., Collopy, F., & Ungar, L.H. (1992). Neural network forecasting of short, noisy time series. Computers and Chemical Engineering,...
  • Funahashi, K. (1989). On the approximate realization of continuous mappings by neural networks. Neural Networks, 2, 183-192.
  • Goodwin, P., & Lawton R. (1999). On the asymetry of the symmetric MAPE. International Journal of Forecasting, 15, 405-408.
  • Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., & Stahel, W.A. (1986). Robust statistics: The approach based on infl uence functions....
  • Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2, 359-366.
  • Huber, P. (1964). Robust estimation of a location parameter. Annals of Mathematical Statistics, 36, 73-101.
  • Huber, P. (1981). Robust statistics. New York: Wiley.
  • Isserman, A. (1977). The accuracy of population projections for subcounty areas. Journal of the American Institute for Planners, 43, 247-259.
  • Janjarasjitt, S., Scher, M.S., & Loparo, K.A. (2008). Nonlinear dynamical analysis of the neonatal EEG time series: the relationship between...
  • Kaastra, I., & Boyd, M. (1996). Designing a neural network for forecasting financial and economic time series. Neurocomputing, 10, 215-236.
  • Law, R., & Au, N. (1999). A neural network model to forecast Japanese demand for travel to Hong Kong. Tourism Management, 20, 89-97.
  • Lewis, C.D. (1982). Industrial and business forecasting methods. London: Butterworths.
  • Makridakis, S. (1993). Accuracy measures: Theoretical and practical concerns. International Journal of Forecasting, 9, 527-529.
  • Makridakis, S., & Hibon, M. (1995). Evaluating accuracy (or error) measures. INSEAD Working Papers Series 95/18/TM. Fontainebleau, France.
  • Makridakis, S., Wheelwright, S.C., & McGee, V.E. (1983). Forecasting: Methods and Applications. New York: Wiley.
  • Mathews, B.P., & Diamantopoulos, A. (1994). Towards a taxonomy of forecast error measures: A factor-comparative investigation of forecast...
  • Miller, R.G. (1986). Beyond ANOVA. Basics of applied statistics. Nueva Cork: John Wiley and Sons.
  • Montaño, J.J., Palmer, A., & Muñoz, P. (2011). Artificial neural networks applied to forecasting time series. Psicothema, 23, 322-329.
  • Murdock, S., Leistritz, L., Hamm, R., Hwang, S., & Parpia, B. (1984). An assessment of the accuracy of a regional economic-demographic...
  • National Research Council (1980). Estimating population and income for small places. Washington, DC: National Academy Press.
  • Palmer, A., Montaño, J.J., & Franconetti, F.J. (2008). Sensitivity analysis applied to artificial neural networks for forecasting time...
  • Pao, H.T. (2006). Modeling and forecasting the energy consumption in Taiwan using artificial neural networks. The Journal of American Academy...
  • Pattie, D.C., & Snyder, J. (1996). Using a neural network to forecast visitor behavior. Annals of Tourism Research, 23, 151-164.
  • Pridemore, W.A., & Chamlin, M.B. (2006). A time-series analysis of the impact of heavy drinking on homicide and suicide mortality in Russia,...
  • Ren, L., & Glasure, Y. (2009). Applicability of the revised mean absolute percentage errors (MAPE) approach to some popular normal and...
  • Sears, C., Davis, T., Guydish, J., & Gleghorn, A. (2009). Investigating the effects of San Francisco's treatment on demand initiative...
  • Sharda, R., & Patil, R. (1990). Neural networks as forecasting experts: An empirical test. Proceedings of the 1990 International Joint...
  • Smith, S. (1987). Tests of accuracy and bias for county, population projections. Journal of the American Statistical Association, 82, 991-1003.
  • Smith, S., & Sincich, T. (1990). On the relationship between length of base period and population forecast errors. Journal of the American...
  • Smith, S., & Sincich, T. (1992). Evaluating the forecast accuracy and bias of alternative projections for states. International Journal...
  • Swanson, D.A., Tayman, J., & Barr, C.F. (2000). A note on the measurement of accuracy for subnational demographic estimates. Demography,...
  • Tang, Z., Almeida, C., & Fishwick, P.A. (1991). Time series forecasting using neural networks vs. Box-Jenkins methodology. Simulation,...
  • Tayman, J., & Swanson, D.A. (1999). On the validity of MAPE as a measure of population forecast accuracy. Population Research and Policy...
  • Tayman, J., Schafer, E., & Carter, L. (1998). The role of population size in the determination and prediction of population forecast errors:...
  • Tayman, J., Swanson, D.A., & Barr, C.F. (1999). In search of the ideal measure of accuracy for subnational demographic forecasts. Population...
  • Tschacher, W., & Ramseyer, F. (2009). Modeling psychotherapy process by time-series panel analysis (TSPA). Psychotherapy Research, 19,...
  • Tukey, J. W. (1977). Exploratory data analysis. Reading, MA: Addison-Wesley.
  • Valiyeva, E., Herrmann, N., Rochon, P.A., Gill, S.S., & Anderson, G.M. (2008). Effect of regulatory warnings on antipsychotic prescription...
  • Wasserman, P.D. (1989). Neural computing: Theory and practice. New York: Van Nostrand Reinhold.
  • Wilcox, R. (1997). Introduction to robust estimation and hypothesis testing. San Diego: Academic Press.