Time Series Forecasting with Recurrent Neural Networks in Presence of Missing Data

Abstract

In many applications, time series forecasting plays an irreplaceable role in time-varying systems such as energy markets, financial markets, and so on. Predicting the dynamic of time-varying systems is essential but is a difficult task because it depends on not only the nature of the system but also on external influences, such as environmental conditions and social and economic status.

Recurrent Neural Networks (RNNs) are a special class of neural networks characterized by the recurrent internal connections, which enable to model the nonlinear dynamical system. Recently, they have been applied in the various forecasting tasks and reported that they outperform the forecast accuracy compared with conventional time series forecasting models. However, there is a limited study of time series forecasting using RNNs in the presence of missing data.

In this thesis, we propose a novel model that utilize Dilated RNN(DRNN) and a modified attention mechanism, focusing on the problem of time series forecasting with missing data. The proposed model outperforms existing models such as AutoRegressive Integrated Moving Average(ARIMA) and Gated Recurrent Unit(GRU), with respect to the forecast accuracy on benchmark datasets.

Besides, we provide a formal description of the learning procedure of RNNs, referred as truncated BPTT(k2, k1), and explain how to construct mini-batches of the training dataset for the forecasting tasks with RNNs, that has not been presented before this work. Discussions and future directions are suggested in five different perspectives at the end.