Exploration of Machine Learning Forecasting Methods in M4 Competition
Abstract
There are so many forecasting algorithms and techniques available. The abilities of Data Mining to obtain and gather data from multiple sources is very useful to researcher, practitioner, business and more. From a long list of forecasting algorithms that have been built throughout the years, it will be exhaustive for someone to go through the list one by one to choose which algorithm to use. With M competition established, there are many more new techniques being innovated each time it is organized. This research aims to compare and contrast the machine learning forecasting techniques that are used in M4 Competition, to understand each technique better and to analyse which is the best technique and why. Three machine learning techniques from M4 Competition was chosen to be compared in this research. Each technique is replicated, trained and tested accordingly. M4 competition dataset is used in this research, with 100,000 time series data and multiple data frequency it is enough to replicate the real-world situation. The results indicate that the three techniques have their strength, with RNN+ES technique on top of it. RNN+ES and CNN-TS performed well in relative to Naive2 benchmark, while k-NS model performed the worst. Further research on the datasets and investigation of each model to further improve its capabilities are needed to improve the performance of the model.
References
Forecasting, vol. 36, no. 1, pp. 7-14, 2020.
[2] R. Fildes and K. Ord, "Forecasting competitions–their role in improving forecasting practice
and research," A companion to economic forecasting, pp. 322-253, 2002.
[3] D. J. Reid, A comparative study of time series prediction techniques on economic data.
University of Nottingham, Library Photographic Unit, 1969.
[4] S. Makridakis, E. Spiliotis, and V. Assimakopoulos, "The M4 Competition: Results, findings,
conclusion and way forward," International Journal of Forecasting, vol. 34, no. 4, pp. 802-808,
2018.
[5] S. Makridakis and M. Hibon, "The M3-Competition: results, conclusions and implications,"
International journal of forecasting, vol. 16, no. 4, pp. 451-476, 2000.
[6] S. Makridakis et al., "The M2-competition: A real-time judgmentally based forecasting study,"
International Journal of Forecasting, vol. 9, no. 1, pp. 5-22, 1993.
[7] S. Makridakis et al., "The accuracy of extrapolation (time series) methods: Results of a
forecasting competition," Journal of forecasting, vol. 1, no. 2, pp. 111-153, 1982.
[8] P. Newbold and C. W. Granger, "Experience with forecasting univariate time series and the
combination of forecasts," Journal of the Royal Statistical Society: Series A (General), vol.
137, no. 2, pp. 131-146, 1974.
[9] G. Libert, "The M‐competition with a fully automatic box–jenkins procedure," Journal of Forecasting, vol. 3, no. 3, pp. 325-328, 1984.
[10] M. Geurts and J. Kelly, "Forecasting demand for special services," International Journal of
Forecasting, vol. 2, no. 3, pp. 90046-4, 1986.
[11] R. Fildes, M. Hibon, S. Makridakis, and N. Meade, "Generalising about univariate forecasting
methods: further empirical evidence," International journal of Forecasting, vol. 14, no. 3, pp.
339-358, 1998.
[12] S. Makridakis and M. Hibon, "Accuracy of forecasting: An empirical investigation," Journal of
the Royal Statistical Society: Series A (General), vol. 142, no. 2, pp. 97-125, 1979.
[13] C. Chatfield, "A personal view of the M2-Competition," International Journal of Forecasting,
vol. 9, no. 1, pp. 23-24, 1993.
[14] R. Fildes and S. Makridakis, "The impact of empirical accuracy studies on time series analysis
and forecasting," International Statistical Review/Revue Internationale de Statistique, pp.
289-308, 1995.
[15] A. J. Koning, P. H. Franses, M. Hibon, and H. O. Stekler, "The M3 competition: Statistical
tests of the results," International Journal of Forecasting, vol. 21, no. 3, pp. 397-409, 2005.
[16] R. J. Hyndman and A. B. Koehler, "Another look at measures of forecast accuracy,"
International journal of forecasting, vol. 22, no. 4, pp. 679-688, 2006.
[17] R. J. Hyndman and G. Athanasopoulos, Forecasting: principles and practice. OTexts, 2018.
[18] S. Makridakis, S. C. Wheelwright, and R. J. Hyndman, Forecasting methods and applications.
John wiley & sons, 2008.
[19] S. Wheelwright, S. Makridakis, and R. J. Hyndman, Forecasting: methods and applications.
John Wiley & Sons, 1998.
[20] E. S. Gardner Jr, "Exponential smoothing: The state of the art—Part II," International journal
of forecasting, vol. 22, no. 4, pp. 637-666, 2006.
[21] E. S. Gardner Jr, "Exponential smoothing: The state of the art," Journal of forecasting, vol. 4,
no. 1, pp. 1-28, 1985.
[22] R. J. Hyndman, A. B. Koehler, R. D. Snyder, and S. Grose, "A state space framework for
automatic forecasting using exponential smoothing methods," International Journal of
forecasting, vol. 18, no. 3, pp. 439-454, 2002.
[23] R. J. Hyndman and Y. Khandakar, Automatic time series for forecasting: the forecast package
for R (no. 6/07). Monash University, Department of Econometrics and Business Statistics,
2007.
[24] S. Smyl, "A hybrid method of exponential smoothing and recurrent neural networks for time
series forecasting," International Journal of Forecasting, vol. 36, no. 1, pp. 75-85, 2020.
[25] I. Taleb, H. T. El Kassabi, M. A. Serhani, R. Dssouli, and C. Bouhaddioui, "Big data quality:
A quality dimensions evaluation," in 2016 Intl IEEE Conferences on Ubiquitous Intelligence &
Computing, Advanced and Trusted Computing, Scalable Computing and Communications,
Cloud and Big Data Computing, Internet of People, and Smart World Congress
(UIC/ATC/ScalCom/CBDCom/IoP/SmartWorld), 2016, pp. 759-765: IEEE.
[26] S. Makridakis, E. Spiliotis, and V. Assimakopoulos, "The M4 Competition: 100,000 time series
and 61 forecasting methods," International Journal of Forecasting, vol. 36, no. 1, pp. 54-74,
2020.
