Deep Learning for Time Series Forecasting: A Survey
Abstract
Time series forecasting has become a very intensive field of research, which is even increasing in recent years. Deep neural networks have proved to be powerful and are achieving high accuracy in many application fields. For these reasons, they are one of the most widely used methods of machine learning to solve problems dealing with big data nowadays. In this work, the time series forecasting problem is initially formulated along with its mathematical fundamentals. Then, the most common deep learning architectures that are currently being successfully applied to predict time series are described, highlighting their advantages and limitations. Particular attention is given to feed forward networks, recurrent neural networks (including Elman, long-short term memory, gated recurrent units, and bidirectional networks), and convolutional neural networks. Practical aspects, such as the setting of values for hyper-parameters and the choice of the most suitable frameworks, for the successful application of deep learning to time series are also provided and discussed. Several fruitful research fields in which the architectures analyzed have obtained a good performance are reviewed. As a result, research gaps have been identified in the literature for several domains of application, thus expecting to inspire new and better forms of knowledge.
Get full access to this article
View all access and purchase options for this article.
References
1. Plageras AP, Psannis KE, Stergiou C, et al. Efficient IoT-based sensor big data collection-processing and analysis in smart buildings. Future Gener Comput Syst. 2018; 82:349–357.
2. Patil HP, Atique M. CDNB: CAVIAR-dragonfly optimization with naive bayes for the sentiment and affect analysis in social media. Big Data. 2020; 8:107–124.
3. Gama J. Knowledge discovery from data streams. UK: Chapman & Hall/CRC, 2010.
4. Al-Jarrah OY, Yoo PD, Muhaidat S, et al. Efficient machine learning for big data: A review. Big Data Res. 2015; 2:87–93.
5. Dhar V, Sun C, Batra P. Transforming finance into vision: Concurrent financial time series as convolutional net. Big Data. 2019; 7:276–285.
6. Nguyen G, Dlugolinsky S, Bobák M, et al. Machine learning and deep learning frameworks and libraries for large-scale data mining: A survey. Artif Intell Rev. 2019; 52:77–124.
7. Maji P, Mullins R. On the reduction of computational complexity of deep convolutional neural networks. Entropy. 2018; 20:305.
8. Schmidhuber J. Deep learning in neural networks: An overview. Neural Netw. 2015; 61:85–117.
9. Makridakis S, Wheelwright SC, Hyndman RJ. Forecasting methods and applications. USA: John Wiley and Sons, 2008.
10. Chatfield C. The analysis of time series: An introduction. UK: Chapman & Hall/CRC, 2003.
11. Box GEP, Jenkins GM. Time series analysis: forecasting and control. USA: John Wiley and Sons, 2008.
12. Martínez-Álvarez F, Troncoso A, Asencio-Cortés G, Riquelme JC. A survey on data mining techniques applied to electricity-related time series forecasting. Energies, 2015; 8:13162–13193.
13. Zhang Q, Yang LT, Chen Z, et al. A survey on deep learning for big data. Inf Fusion, 2018; 42:146–157.
14. Fawaz HI, Forestier G, Weber J, et al. Deep learning for time series classification: A review. Data Min Knowl Discov, 2019; 33:917–963.
15. Bagnall A, Lines J, Vickers W, et al. 2017. The UEA & UCR time series classification repository. Available online at www.timeseriesclassification.com. (last accessed on April30, 2020).
16. Mayer R, Jacobsen HA. Scalable deep learning on distributed infrastructures: challenges, techniques, and tools. ACM Comput Surv, 2020; 53:Article 3.
17. Buuren S. Flexible imputation of missing data. UK: Chapman & Hall/CRC, 2012.
18. Maronna RA, Martin RD, Yohai VJ. Robust statistics: theory and methods. USA: Wiley, 2006.
19. Fu TC. A review on time series data mining. Eng Appl Artif Intell. 2011; 24:164–181.
20. Shumway RH, Stoffer DS. Time series analysis and its applications (with R examples). USA: Springer, 2011.
21. Hyndman RJ, Athanasopoulos G. Forecasting: principles and practice. Australia: Otexts, 2018.
22. Wang X, Kang Y, Hyndman RJ, et al. Distributed ARIMA models for ultra-long time series. arXiv e-prints, arXiv: 2007.09577, 2020.
23. Rakthanmanon T, Campana B, Mueen A, et al. Addressing big data time series: Mining trillions of time series subsequences under dynamic time warping. ACM Trans Knowl Discov Data, 2013; 7:10.
24. Torres JF, Galicia A, Troncoso A, et al. A scalable approach based on deep learning for big data time series forecasting. Integr Comput Aided Eng, 2018; 25:335–348.
25. Galicia A, Talavera-Llames RL, Troncoso A, et al. Multi-step forecasting for big data time series based on ensemble learning. Knowl Based Syst, 2019; 163:830–841.
26. Talavera-Llames R, Pérez-Chacón R, Troncoso A, et al. Big data time series forecasting based on nearest neighbors distributed computing with spark. Knowl Based Syst, 2018; 161:12–25.
27. Talavera-Llames R, Pérez-Chacón R, Troncoso A, Martínez-Álvarez F. MV-kWNN: A novel multivariate and multi-output weighted nearest neighbors algorithm for big data time series forecasting. Neurocomputing, 2019; 353:56–73.
28. Pérez-Chacón R, Asencio-Cortés G, Martínez Álvarez F, et al. Big data time series forecasting based on pattern sequence similarity and its application to the electricity demand. Inf Sci, 2020; 540:160–174.
29. Rumelhart D, Hinton G, Williams R. Long short-term memory. Nature, 1986; 323:533–536.
30. Elman JL. Finding structure in time. Energy Rep, 1990; 14:179–211.
31. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput, 1997; 9:1735–1780.
32. Chung J, Gulcehre C, Cho K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling. In: Proceedings of the Neural Information Processing Systems, Canada, 2014. pp. 1–12.
33. Cho K, Merrienboer BV, Bahdanau D, et al. On the properties of neural machine translation: encoder-decoder approaches. In: Proceedings of SSST-8. Qatar, 2014. pp. 103–111.
34. Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. In: Proceedings of the International Conference on Learning Representations, 2015, pp. 149–158.
35. Xu K, Ba J, Kiros R, et al. Show, attend and tell: Neural image caption generation with visual attention. In: Proceedings of the International Conference on Machine Learning, 2015, pp. 2048–2057.
36. Fukushima K. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern, 1980; 36:193–202.
37. Zhang W, Hasegawa A, Matoba O, et al. Shift-invariant neural network for image processing: Learning and generalization. Appl Artif Neural Netw III, 1992; 1709:257–268.
38. Alla S, Adari SK. Beginning anomaly detection using Python-based deep learning. USA: Apress, 2019.
39. Abadi M, Agarwal A, Barham P, et al. 2015. TensorFlow: large-scale machine learning on heterogeneous systems. Available online at http://tensorflow.org. (last accessed on April30, 2020).
40. Candel A, LeDell E, Arora A, et al. 2015. Deep learning with h2o. Available online at http://h2o.ai/resources. (last accessed on April30, 2020).
41. Eclipse Deeplearning4j development team. 2016. DL4J: deep learning for Java. Available online at https://github.com/eclipse/deeplearning4j. (last accessed on April30, 2020).
42. Paszke A, Gross S, Massa F, et al. Pytorch: an imperative style, highperformance deep learning library. In: Proceedings of the Advances in Neural Information Processing Systems. Vol. 32, 2019. pp. 8026–8037.
43. Jia Y, Shelhamer E, Donahue J, et al. Caffe: Convolutional architecture for fast feature embedding. arXiv e-prints, arXiv: 1408.5093, 2014.
44. Intel Nervana systems. Neon deep learning framework 2017. Available online at https://github.com/NervanaSystems/neon, 2017. (last accessed on April30, 2020).
45. Tokui S, Okuta R, Akiba T, et al. Chainer: A deep learning framework for accelerating the research cycle. In: Proceedings of International Conference on Knowledge Discovery and Data Mining 2019, pp. 2002–2011.
46. Theano Development Team. Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints, arXiv: 1605.02688, 2016.
47. Tianqi C, Li M, Li Y, et al. Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv e-prints arXiv: 1512.01274, 2015.
48. Bai J, Lu F, Zhang K, et al. 2019. Onnx: Open neural network exchange. Available online at https://github.com/onnx/onnx. (last accessed on June15, 2020).
49. Seide F, Agarwal A. CNTK: Microsoft's open-source deep-learning toolkit. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. USA, 2016, pp. 2135–2135.
50. Chollet F. 2015. Keras. Available online at https://keras.io
51. DeepMind revision. Sonnet, 2019.
52. Guo J, He H, He T, et al. Gluoncv and gluonnlp: Deep learning in computer vision and natural language processing. J Mach Learn Res, 2020; 21:1–7.
53. Choi D, Shallue CJ, Nado Z, et al. On empirical comparisons of optimizers for deep learning. arXiv e-prints, arXiv: 1910.05446, 2019.
54. You K, Long M, Wang J, Jordan MI. How does learning rate decay help modern neural networks? In Proceedings of the International Conference on Learning Representations, 2019. pp. 1–14.
55. Sinha S, Singh TN, Singh V, Verma A. Epoch determination for neural network by self-organized map (SOM). Comput Geosci, 2010 14:199–206.
56. Masters D, Luschi C. Revisiting small batch training for deep neural networks. arXiv e-prints, arXiv: 1804.07612, 2018.
57. Shafi I, Ahmad J, Shah SI, et al. Impact of varying neurons and hidden layers in neural network architecture for a time frequency application. In: Proceedings of the IEEE International Multitopic Conference. USA, 2006. pp. 188–193.
58. Ding B, Qian H, Zhou J. Activation functions and their characteristics in deep neural networks. In: Proceedings of the Chinese Control and Decision Conference. USA, 2018. pp. 1836–1841.
59. Sutskever I, Martens J, Dahl G, et al. On the importance of initialization and momentum in deep learning. In: Proceedings of the International Conference on Machine Learning, 2013. pp. 1139–1147.
60. Kumar SK. On weight initialization in deep neural networks. arXiv e-prints, arXiv: 1704.08863, 2017.
61. Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res, 2014; 15:1929–1958.
62. Ng AY. Feature selection, l1 vs. l2 regularization, and rotational invariance. In: Proceedings of the ACM International Conference on Machine Learning. USA, 2004. pp. 78–85.
63. Mairal J, Koniusz P, Harchaoui Z, Schmid C. Convolutional kernel networks. In: Proceedings of the Neural Information Processing Systems. USA, 2014. pp. 1–9.
64. Zaniolo L, Marques O. On the use of variable stride in convolutional neural networks. Multimed Tools Appl. 2020; 79:13581–13598.
65. Dwarampudi M, Reddy NVS. Effects of padding on LSTMs and CNNs. arXiv e-prints, arXiv: 1903.07288, 2019.
66. Zhu H, An Z, Yang C, et al. Rethinking the number of channels for the convolutional neural network. arXiv e-prints, arXiv: 1909.01861, 2019.
67. Scherer F, Müller A, Behnke S. Evaluation of pooling operations in convolutional architectures for object recognition. In: Proceedings of Artificial Neural Networks. USA, 2010. pp. 92–101.
68. Ma B, Li X, Xia Y, et al. Autonomous deep learning: A genetic DCNN designer for image classification. Neurocomputing, 2020; 379:152–161.
69. Itano F, De-Abreu-De-Sousa MA, Del-Moral-Hernandez E. Extending MLP ANN hyper-parameters optimization by using genetic algorithm. In: Proceedings of the IEEE International Joint Conference on Neural Networks. USA, 2018. pp. 1–8.
70. Kennedy K, Eberhart R. Particle swarm optimization. In: Proceedings of International Conference on Neural Networks. Vol. 4, USA, 1995. pp. 1942–1948.
71. Stanley KO, Miikkulainen R. Evolving neural networks through augmenting topologies. Evol Comput. 2002; 10:99–127.
72. Martínez-Álvarez F, Asencio-Cortés G, Torres JF, et al. Coronavirus optimization algorithm: A bioinspired metaheuristic based on the COVID-19 Propagation Model. Big Data, 2020; 8:308–322.
73. Ranjit MP, Ganapathy G, Sridhar K, et al. Efficient deep learning hyperparameter tuning using cloud infrastructure: intelligent distributed hyperparameter tuning with bayesian optimization in the cloud. In: Proceedings of International Conference on Cloud Computing. USA, 2019. pp. 520–522.
74. Bergstra J, Yamins D, Cox DD. Making a science of model search: hyperparameter optimization in hundreds of dimensions for vision architectures. In: Proceedings of the International Conference on International Conference on Machine Learning, 2013. pp. 115–123.
75. Camero A, Toutouh J, Alba E. Dlopt: Deep learning optimization library. arXiv e-prints, arXiv: 1807.03523, 2018.
76. Autonomio. Talos. Available online at http://github.com/autonomio/talos, 2019.
77. Balandat M, Karrer B, Jiang DR, et al. BoTorch: Programmable Bayesian Optimization in PyTorch. arXiv e-prints, arXiv: 1910.06403, 2019.
78. Eggensperger K, Feurer M, Hutter F, et al. Towards an empirical foundation for assessing bayesian optimization of hyperparameters. In: Proceedings of the Neural Information Processing Systems. USA, 2013. pp. 1–5.
79. Wang YE, Wei GY, Brooks D. Benchmarking TPU, GPU, and CPU platforms for deep learning. arXiv e-prints, arXiv: 1907.10701, 2019.
80. Yu D, Wang Y, Liu H, et al. System identification of PEM fuel cells using an improved Elman neural network and a new hybrid optimization algorithm. Energy Rep. 2019; 5:1365–1374.
81. Zheng Y, Yao Z, Zhou H, et al. Power generation forecast of top gas recovery turbine unit based on Elman model. In: Proceedings of the IEEE Chinese Control Conference. USA, 2018. pp. 7498–7501.
82. Ruiz LGB, Rueda R, Cuéllar MP, et al. Energy consumption forecasting based on Elman neural networks with evolutive optimization. Expert Syst Appl. 2018; 92:380–389.
83. Wang J, Lv Z, Liang Y, et al. Fouling resistance prediction based on GA–Elman neural network for circulating cooling water with electromagnetic anti-fouling treatment. J Energy Inst. 2019; 92:1519–1526.
84. Li W, Jiao Z, Du L, et al. An indirect RUL prognosis for lithium-ion battery under vibration stress using Elman neural network. Int J Hydrog Energy. 2019; 44:12270–12276.
85. Yu Y, Wang X, Bründlinger R. Improved Elman neural network short-term residents load forecasting considering human comfort index. J Electr Eng Technol. 2019; 14:2315–2322.
86. Li D, Wang H, Zhang Y, et al. Power grid load state information perception forecasting technology for battery energy storage system based on Elman neural network. In: Proceedings of information Technology, Networking, Electronic and Automation Control Conference. USA, 2019. pp. 914–917.
87. Abdel-Nasser M, Mahmoud K. Accurate photovoltaic power forecasting models using deep LSTM-RNN. Neural Comput Appl. 2019; 31:2727–2740.
88. Khodabakhsh A, Ari I, Bakır M, et al. Forecasting multivariate time-series data using LSTM and mini-batches. In: Proceedings of Data Engineering and Communications Technologies, 2020. pp. 121–129.
89. Gao M, Li J, Hong F, et al. Day-ahead power forecasting in a large-scale photovoltaic plant based on weather classification using LSTM. Energy. 2019; 187:115838.
90. Muzaffar S, Afshari A. Short-term load forecasts using LSTM networks. Proc Energy Proc. 2019; 158:2922–2927.
91. Song X, Liu Y, Xue L, et al. Time-series well performance prediction based on long short-term memory (LSTM) neural network model. J Pet Sci Eng. 2020; 186:106682.
92. Wang JQ, Du Y, Wang J. LSTM based long-term energy consumption prediction with periodicity. Energy. 2020; 197:117197.
93. Gokhan A, Yilmaz E, Unel M, et al. Estimating soot emission in diesel engines using gated recurrent unit networks. IFAC Papers Online. 2019; 52:544–549.
94. Wu W, Liao W, Miao J, et al. Using gated recurrent unit network to forecast short-term load considering impact of electricity price. Proc Energy Proc. 2019; 158:3369–3374.
95. Tang X, Dai Y, Wang T, et al. Short-term power load forecasting based on multi-layer bidirectional recurrent neural network. IET Gener Transm Distrib. 2019; 13:3847–3854.
96. Shao Z, Zheng Q, Yang S, et al. Modeling and forecasting the electricity clearing price: A novel BELM based pattern classification framework and a comparative analytic study on multi-layer BELM and LSTM. Energy Econ. 2020; 86:104648.
97. Qiu X, Ren Y, Suganthan PN, et al. Empirical mode decomposition based ensemble deep learning for load demand time series forecasting. Appl Soft Comput J. 2017; 54:246–255.
98. Qiu X, Zhang L, Ren Y, et al. Ensemble deep learning for regression and time series forecasting. In: Proceedings of the IEEE Symposium Series on Computational Intelligence in Ensemble Learning. USA, 2014. pp. 1–6.
99. Manohar M, Koley E, Ghosh S, et al. Spatio-temporal information based protection scheme for PV integrated microgrid under solar irradiance intermittency using deep convolutional neural network. Int J Electr Power Energy Syst. 2020; 116:105576.
100. Mishra K, Basu S, Maulik U. DaNSe: A dilated causal convolutional network based model for load forecasting. Lect Notes Comput Sci. 2019; 11941:234–241.
101. Qiao W, Yang Z, . Forecast the electricity price of U.S. using a wavelet transform-based hybrid model. Energy, 2020; 193:116704.
102. Ji L, Zou Y, He K, et al. Carbon futures price forecasting based with ARIMA-CNN-LSTM model. Proc Comput Sci. 2019; 162:33–38.
103. Kim TY, Cho SB. Predicting residential energy consumption using CNN-LSTM neural networks. Energy. 2019; 182:72–81.
104. Shen M, Xu Q, Wang K, et al. Short-term bus load forecasting method based on cnn-gru neural network. Lect Notes Electr Eng, 2020; 585:711–722.
105. AlKandari M, Ahmad I. Solar power generation forecasting using ensemble approach based on deep learning and statistical methods. Appl Comput Inf. 2020; 6:1–20.
106. Kong Z, Tang B, Deng L, et al. Condition monitoring of wind turbines based on spatio-temporal fusion of SCADA data by convolutional neural networks and gated recurrent units. Renew Energy, 2020; 146:760–768.
107. Wang S, Zhang Z, Ren Y, et al. UAV photogrammetry and AFSA-Elman neural network in slopes displacement monitoring and forecasting. KSCE J Civil Eng. 2020; 24:19–29.
108. Sarkar S, Lore KG, Sarkar S, et al. Early detection of combustion instability from hi-speed flame images via deep learning and symbolic time series analysis. In: Proceedings of the Annual Conference of the Prognostics and Health Management Society. USA, 2015. pp. 353–362.
109. Ma X, Dai Z, He Z, et al. Learning traffic as images: A deep convolutional neural network for large-scale transportation network speed prediction. Sensors, 2017; 17:818.
110. Chen W, Shi K. A deep learning framework for time series classification using relative position matrix and convolutional neural network. Neurocomputing, 2019; 359:384–394.
111. Ienco D, Interdonato R, Gaetano R, et al. Combining Sentinel-1 and Sentinel-2 satellite image time series for land cover mapping via a multi-source deep learning architecture. J Photogrammetry Remote Sens. 2019; 158:11–22.
112. Martínez-Arellano G, Terrazas G, Ratchev S. Tool wear classification using time series imaging and deep learning. Int J Adv Manuf Technol. 2019; 104:3647–3662.
113. Wu W, Zhang J, Xie H, et al. Automatic detection of coronary artery stenosis by convolutional neural network with temporal constraint. Comput Biol Med, 2020; 118:103657.
114. Miao Y, Han J, Gao Y, Zhang B. ST-CNN: Spatial-Temporal Convolutional Neural Network for crowd counting in videos. Pattern Recogn Lett. 2019; 125:113–118.
115. Feng S. Dynamic facial stress recognition in temporal convolutional network. In: Proceedings of the Communications in Computer and Information Science. USA, 2019. pp. 698–706.
116. Zhang Y, Kampffmeyer M, Liang X, et al. Dilated temporal relational adversarial network for generic video summarization. Multimed Tools Appl. 2019; 78:35237–35261.
117. Interdonato R, Ienco D, Gaetano R, et al. DuPLO: A DUal view Point deep Learning architecture for time series classificatiOn. ISPRS J Photogramm Remote Sens. 2019; 149:91–104.
118. Yan H, Ouyang H. Financial time series prediction based on deep learning. Wireless Pers Commun. 2018; 102:683–700.
119. Sismanoglu G, Onde M, Kocer F, et al. Deep learning based forecasting in stock market with big data analytics. In: Proceedings of the IEEE Scientific Meeting on Electrical-Electronics and Biomedical Engineering and Computer Science. USA, 2019. pp. 10057–10059.
120. Jayanth BA, Harish RDS, Nair BB. Applicability of deep learning models for stock price forecasting an empirical study on bankex data. Proc Comput Sci. 2018; 143:947–953.
121. Jiang M, Liu J, Zhang L, et al. An improved stacking framework for stock index prediction by leveraging tree-based ensemble models and deep learning algorithms. Physica A, 2020; 541:122272.
122. Wu W, Wang Y, Fu J, et al. Preliminary study on interpreting stock price forecasting based on tree regularization of GRU. In: Proceedings of Communications in Computer and Information Science. Vol. 1059, USA, 2019. pp. 476–487.
123. Orimoloye LO, Sung MC, Ma T, et al. Comparing the effectiveness of deep feedforward neural networks and shallow architectures for predicting stock price indices. Expert Syst Appl. 2020; 139:112828.
124. Makarenko AV. Deep learning algorithms for estimating Lyapunov exponents from observed time series in discrete dynamic systems. In: Proceedings of International Conference Stability and Oscillations of Nonlinear Control Systems. USA, 2018. pp. 1–4.
125. Dingli A, Fournier KS. Financial time series forecasting—a deep learning approach. Int J Mach Learn Comput. 2017; 7:118–122.
126. Kelotra A, Pandey P. Stock market prediction using Optimized Deep-ConvLSTM Model. Big Data, 2020; 8:5–24.
127. Ni L, Li Y, Wang X, et al. Forecasting of Forex time series data based on deep learning. Proc Comput Sci. 2019; 147:647–652.
128. Bao W, Yue J, Rao Y. A deep learning framework for financial time series using stacked autoencoders and long-short term memory. PLos One. 2017; 12: e0180944.
129. Munkhdalai L, Li M, Theera-Umpon N, et al. VAR-GRU: A hybrid model for multivariate financial time series prediction. Lect Notes Artif Intell. 2020; 12034:322–332.
130. Chen CT, Chiang LK, Huang YC, et al. Forecasting interaction of exchange rates between fiat currencies and cryptocurrencies based on deep relation networks. In: Proceedings of the IEEE International Conference on Agents. USA, 2019. pp. 69–72.
131. Berradi Z, Lazaar M. Integration of principal component analysis and recurrent neural network to forecast the stock price of Casablanca stock exchange. Proc Comput Sci, 2019; 148:55–61.
132. Wang Q, Xu W, Huang X, et al. Enhancing intraday stock price manipulation detection by leveraging recurrent neural networks with ensemble learning. Neurocomputing, 2019; 347:46–58.
133. Long W, Lu Z, Cui L. Deep learning-based feature engineering for stock price movement prediction. Knowl Based Syst. 2019; 164:163–173.
134. Liu H, Tian HQ, Liang XF, et al. Wind speed forecasting approach using secondary decomposition algorithm and Elman neural networks. Appl Energy. 2015; 157:183–194.
135. Yu C, Li Y, Xiang H, et al. Data mining-assisted short-term wind speed forecasting by wavelet packet decomposition and Elman neural network. J Wind Eng Indus Aerodyn. 2018; 175:136–143.
136. Liu H, Wei MX, Fei LY. Wind speed forecasting method based on deep learning strategy using empirical wavelet transform, long short term memory neural network and Elman neural network. Energy Conv Manage. 2018; 156:498–514.
137. Zhang Y, Pan G. A hybrid prediction model for forecasting wind energy resources. Environ Sci Pollut Res. 2020; 27:19428–19446.
138. Zhang L, Xie Y, Chen A, et al. A forecasting model based on enhanced Elman neural network for air quality prediction. Lect Notes Electr Eng. 2019; 518:65–74.
139. Huang Y, Shen L. Elman neural network optimized by firefly algorithm for forecasting China's carbon dioxide emissions. Commun Comput Inf Sci. 2018; 951:36–47.
140. Xiao D, Hou S, Li WZ, et al. Hourly campus water demand forecasting using a hybrid EEMD-Elman neural network model. In: Sustainable Development of Water Resources and Hydraulic Engineering in China. Environmental Earth Sciences. USA, 2019. pp. 71–80.
141. Shen W, Fu X, Wang R, et al. A prediction model of NH3 concentration for swine house in cold region based on empirical mode decomposition and Elman neural network. Inf Proc Agric. 2019; 6:297–305.
142. Wan X, Yang Q, Jiang P, et al. A hybrid model for real-time probabilistic flood forecasting using Elman neural network with heterogeneity of error distributions. Water Resour Manage. 2019; 33:4027–4050.
143. Wan H, Guo S, Yin K, et al. CTS-LSTM: LSTM-based neural networks for correlatedtime series prediction. Knowl Based Syst. 2019; 191:105239.
144. Freeman BS, Taylor G, Gharabaghi B, et al. Forecasting air quality time series using deep learning. J Air Waste Manage Assoc. 2018; 68:866–886.
145. De-Melo GA, Sugimoto DN, Tasinaffo PM, et al. A new approach to river flow forecasting: LSTM and GRU multivariate models. IEEE Latin Am Trans. 2019; 17:1978–1986.
146. Chen J, Zeng GQ, Zhou W, et al. Wind speed forecasting using nonlinear-learning ensemble of deep learning time series prediction and extremal optimization. Energy Conv Manage. 2018; 165:681–695.
147. Niu Z, Yu Z, Tang W, et al. Wind power forecasting using attention-based gated recurrent unit network. Energy. 2020; 196:117081.
148. Li W, Wu H, Zhu N, et al. Prediction of dissolved oxygen in a fishery pond based on gated recurrent unit (GRU). Inf Process Agric. 2020, In press.
149. Peng Z, Peng S, Fu L, et al. A novel deep learning ensemble model with data denoising for short-term wind speed forecasting. Energy Convers Manage. 2020; 207:112524.
150. Jin X, Yu X, Wang X, et al. Prediction for time series with CNN and LSTM. Lect Notes Electr Eng, 2020; 582:631–641.
151. Maqsood H, Mehmood I, Maqsood M, et al. A local and global event sentiment based efficient stock exchange forecasting using deep learning. Int J Inf Manage. 2020; 50:432–451.
152. O'Shea TJ, Roy T, Clancy TC. Over-the-air deep learning based radio signal classification. IEEE J Select Topics Signal Process. 2018; 12:168–179.
153. Zhang Y, Thorburn PJ, Fitch P. Multi-task temporal convolutional network for predicting water quality sensor data. In: Proceedings of Neural Information Processing Communications in Computer and Information Science. USA, 2019. pp. 122–130.
154. Sun Y, Zhao Z, Ma X, et al. Short-timescale gravitational microlensing events prediction with ARIMA-LSTM and ARIMA-GRU hybrid model. Lect Notes Comput Sci. 2019; 11473:224–238.
155. Liu H, Mi X, Li Y, et al. Smart wind speed deep learning based multi-step forecasting model using singular spectrum analysis, convolutional gated recurrent unit network and support vector regression. Renew Energy. 2019; 143:842–854.
156. Li X, Zhang L, Wang Z, et al. Remaining useful life prediction for lithium-ion batteries based on a hybrid model combining the long short-term memory and Elman neural networks. J Energy Storage. 2019; 21:510–518.
157. Yang L, Wang F, Zhang J, et al. Remaining useful life prediction of ultrasonic motor based on Elman neural network with improved particle swarm optimization. Measurement. 2019; 143:27–38.
158. Rashid KM, Louis J. Times-series data augmentation and deep learning for construction equipment activity recognition. Adv Eng Inform. 2019; 42:100944.
159. Huang X, Zanni-Merk C, Crémilleux B. Enhancing deep learning with semantics: An application to manufacturing time series analysis. Proc Comput Sci. 2019; 159:437–446.
160. Mehdiyev N, Lahann J, Emrich A, et al. Time series classification using deep learning for process planning: A case from the process industry. Proc Comput Sci, 2017; 114:242–249.
161. Wang S, Chen J, Wang H, et al. Degradation evaluation of slewing bearing using HMM and improved GRU. Measurement. 2019; 146:385–395.
162. Wang J, Yan J, Li C, et al. Deep heterogeneous GRU model for predictive analytics in smart manufacturing: Application to tool wear prediction. Comput Indus. 2019; 111:1–14.
163. Bohan H, Yun B. Traffic flow prediction based on BRNN. In: Proceedings of the IEEE International Conference on Electronics Information and Emergency Communication. USA, 2019. pp. 320–323.
164. Pasias A, Vafeiadis T, Ioannidis D, et al. Forecasting bath and metal height features in electrolysis process. In: Proceedings of the International Conference on Distributed Computing in Sensor Systems. 2019. pp. 312–317.
165. Jiang P, Chen C, Liu X. Time series prediction for evolutions of complex systems: A deep learning approach. In: Proceedings of rhe IEEE International Conference on Control and Robotics Engineering, 2016. pp. 1–6.
166. Canizo M, Triguero I A Conde, et al. Multi-head CNN–RNN for multi-time series anomaly detection: An industrial case study. Neurocomputing. 2019; 363:246–260.
167. Kuang L, Hua C, Wu J, et al. Traffic volume prediction based on multi-sources GPS trajectory data by temporal convolutional network. Mobile Netw Appl. 2020; 25:1–13.
168. Wu P, Sun J, Chang X, et al. Data-driven reduced order model with temporal convolutional neural network. Comput Methods Appl Mech Eng. 2020; 360:112766.
169. Varona B, Monteserin A, Teyseyre A. A deep learning approach to automatic road surface monitoring and pothole detection. Pers Ubiquitous Comput. 2020; 24:519–534.
170. Cai M, Pipattanasomporn M, Rahman S. Day-ahead building-level load forecasts using deep learning vs. traditional time-series techniques. Appl Energy. 2019; 236:1078–1088.
171. da Silva DB, Schmidt D, da Costa CA, da Rosa Righi R, Eskofier B. Deepsigns: A predictive model based on deep learning for the early detection of patient health deterioration. Expert Syst Appl. 2021; 165:113905.
172. Yu W, Kim Y, Mechefske C. Remaining useful life estimation using a bidirectional recurrent neural network based autoencoder scheme. Mech Syst Signal Process, 2019; 129:764–780.
173. Bui C, Pham N, Vo A, Tran A, Nguyen A, Le T. Time series forecasting for healthcare diagnosis and prognostics with the focus on cardiovascular diseases. In: Proceedings of the International Conference on the Development of Biomedical Engineering in Vietnam. USA, 2018. pp. 809–818.
174. Liu Y, Huang Y, Wang J, et al. Detecting premature ventricular contraction in children with deep learning. J Shanghai Jiaotong Univ. 2018; 23:66–73.
175. Hoppe E, Körzdörfer G, Würfl T, et al. Deep learning for magnetic resonance fingerprinting: A new approach for predicting quantitative parameter values from time series. In: Studies in Health Technology and Informatics, Vol. 243. Netherlands: IOS Press, 2017. pp. 202–206.
176. Chambon S, Galtier MN, Arnal PJ, et al. A deep learning architecture for temporal sleep stage classification using multivariate and multimodal time series. IEEE Trans Neural Syst and Rehabil Eng. 2018; 26:758–769.
177. Lauritsen SM, Kalør ME, Kongsgaard EL, et al. Early detection of sepsis utilizing deep learning on electronic health record event sequences. Artif Intell Med. 2020; 104:101820.
178. Chen X, He J, Wu X, et al. Sleep staging by bidirectional long short-term memory convolution neural network. Fut Gen Comput Syst. 2020; 109:188–196.
179. Liang liang M, Fu peng T. Pneumonia incidence rate predictive model of nonlinear time series based on dynamic learning rate BP neural network. In: Proceedings of the Fuzzy Information and Engineering. USA, 2010. pp. 739–749.
180. Sarafrazi S, Choudhari RS, Mehta C, et al. Cracking the “sepsis” code: Assessing time series nature of ehr data, and using deep learning for early sepsis prediction. In: Proceedings of the Computing in Cardiology. UK, 2019. pp. 1–4.
181. Zeroual A, Harrou F, Dairi A, Sun Y. Deep learning methods for forecasting covid-19 time-series data: A comparative study. Chaos Solit Fract. 2020; 140:110121.
182. Zhang X, Shen F, Zhao J, et al. Time series forecasting using GRU neural network with multi-lag after decomposition. Lect Notes Comput Sci. 2017; 10638:523–532.
183. Zhao X, Xia L, Zhang J, et al. Artificial neural network based modeling on unidirectional and bidirectional pedestrian flow at straight corridors. Physica A. 2020; 547:123825.
184. Imamverdiyev Y, Abdullayeva F. Deep learning method for denial of service attack detection based on restricted boltzmann machine. Big Data. 2018; 6:159–169.
185. Munir M, Siddiqui SA, Dengel A, et al. DeepAnT: A deep learning approach for unsupervised anomaly detection in time series. IEEE Access. 2019; 7:1991–2005.
186. Zebin T, Scully PJ, Ozanyan KB. Human activity recognition with inertial sensors using a deep learning approach. In: Proceedings of IEEE Sensors. USA, 2017. pp. 1–3.
187. Bendong Z, Huanzhang L, Shangfeng C, et al. Convolutional neural networks for time series classification. J Syst Eng Electr. 2017; 28:162–169.
188. Jiang W, Wang Y, Tang Y. A sequence-to-sequence transformer premised temporal convolutional network for chinese word segmentation. In: Proceedings of Parallel Architectures, Algorithms and Programming. USA, 2020. pp. 541–552.
189. Shao J, Shen H, Cao Q, et al. Temporal convolutional networks for popularity prediction of messages on social medias. Lect Notes Comput Sci. 2019;:135–147.
190. Chen Y, Kan Y, Chen Y, et al. Probabilistic forecasting with temporal convolutional neural network. Neurocomputing. 2020; 399:491–501.
191. Xi R, Hou M, Fu M, et al. Deep dilated convolution on multimodality time series for human activity recognition. In: Proceedings of the IEEE International Joint Conference on Neural Networks. USA, 2018. pp. 53381–53396.
192. Wang R, Peng C, Gao J, et al. A dilated convolution network-based LSTM model for multi-step prediction of chaotic time series. Comput Appl Math. 2020; 39:1–22.
193. Rodrigues F, Markou I, Pereira FC. Combining time-series and textual data for taxi demand prediction in event areas: A deep learning approach. Inf Fusion. 2019; 49:120–129.
194. Kalinin MO, Lavrova DS, Yarmak AV. Detection of threats in cyberphysical systems based on deep learning methods using multidimensional time series. Autom Control Comput Sci. 2018; 52:912–917.
195. Wang H, Lei Z, Zhang X, et al. A review of deep learning for renewable energy forecasting. Energy Conv Manage. 2019; 198:111799.
196. Hu Z, Tang J, Wang Z, et al. Deep learning for image-based cancer detection and diagnosis—A survey. Pattern Recogn. 2018; 83:134–149.
197. Atto AM, Benoit A, Lambert P. Timed-image based deep learning for action recognition in video sequences. Pattern Recogn. 2020; 104:107353.
198. Sezer OB, Gudelek M, Ozbayoglu AM. Financial time series forecasting with deep learning: A systematic literature review: 2005–2019. Appl Soft Comput. 2020; 90:106181.
199. Shen Z, Zhang Y, Lu J, et al. A novel time series forecasting model with deep learning. Neurocomputing. 2020; 396:302–313.
200. Wang Y, Zhang D, Liu Y, et al. Enhancing transportation systems via deep learning: A survey. Transp Res Part C Emerg Technol. 2019; 99:144–163.
201. Kamilaris A, Prenafeta-Boldú FX. Deep learning in agriculture: A survey. Comput Electr Agric. 2018; 147:70–90.
202. Rui Z, Ruqiang Y, Zhenghua C, et al. Deep learning and its applications to machine health monitoring. Mech Syst Signal Process. 2019; 115:213–237.
203. Mahdavifar S, Ghorbani AA. Application of deep learning to cybersecurity: A survey. Neurocomputing, 2019; 347:149–176.
Cite this article as: Torres JF, Hadjout D, Sebaa A, Martínez-Álvarez F, Troncoso A (2021) Deep learning for time series forecasting: a survey. Big Data 9:1, 3–21, DOI: 10.1089/big.2020.0159.
Cite
Cite
Cite
OR
Download to reference manager
If you have citation software installed, you can download citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: February 5, 2021
Issue published: February 5, 2021
Keywords
PubMed: 33275484
Authors
Metrics and citations
Metrics
Publication usage*
Total views and downloads: 1550
*Publication usage tracking started in December 2016
Publications citing this one
Receive email alerts when this publication is cited
Web of Science: 0
Crossref: 627
- Data-driven greenhouse climate regulation in lettuce cultivation using BiLSTM and GRU predictive control
- TabulaTime: Novel multimodal deep learning for Acute Coronary Syndrome prediction through environmental and clinical data integration
- TempFusion-Nexus: A multimodal fusion-driven adaptive model for significant wave height forecasting
- Compact transformer variants for synthetic time series forecasting
- Point-wise vs. cumulative solar forecasting: From daily operations to long-term planning
- Forecasting time series collections via fuzzy clustering
- Computational Intelligence in Engineering Science
- Enhancing COVID-19 forecasts with a lightweight multi-head depthwise separable convolution network
- Out-of-Distribution Generalization in Time Series: A Survey
- Forecasting-based biomedical time-series data synthesis for open data and robust AI
- View More
Figures and tables
Figures & Media
Tables
View Options
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
I am signed in as:
View my profileSign out
I can access personal subscriptions, purchases, paired institutional access and free tools such as favourite journals, email alerts and saved searches.
loading institutional access options
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
