oa Learning Spatiotemporal Latent Factors of Traffic via a Regularized Tensor Factorization: Imputing Missing Values and Forecasting

Spatiotemporal data related to traffic has become common place due to the wide availability of cheap sensors and the rapid deployment of IoT platforms. Yet, this data suffer several challenges related to sparsity, incompleteness, and noise, which makes traffic analytics difficult. In this paper, we investigate the problem of missing data or noisy information in the context of real-time monitoring and forecasting of traffic congestion for road networks. The road network is represented as a directed graph in which nodes are junctions and edges are road segments. We assume that the city has deployed high-fidelity sensors for speed reading in a subset of edges. Our objective is to infer speed readings for the remaining edges in the network as well as missing values to malfunctioning sensors. We propose a tensor representation for the series of road network snapshots, and develop a regularized factorization method to estimate the missing values, while learning the latent factors of the network. The regularizer, which incorporates spatial properties of the road network, improves the quality of the results. The learned factors along with a graph-based temporal dependency are used in an autoregressive algorithm to predict the future state of the road network with long horizon. Extensive numerical experiments with real traffic data from the cities of Doha(Qatar) and Aarhus (Denmark) demonstrate that the proposed approach is appropriate for imputing missing data and predicting traffic state.Main contributions. The main contributions are:We propose a novel temporal regularized tensor factorization framework (TRTF) for high-dimensional traffic data. TRTF provides a principled approach to account for both the spatial structure and the temporal dependencies.We introduce a novel data-driven graph-based autoregressive model, where the weights are learned from the data. Hence, the regularizer can account for both positive and negative correlations.We show that incorporating temporal embeddings into CP-WOPT leads to accurate multi-step forecasting, compared to state of the art matrix factorization based methods.We conduct extensive experiments on real traffic congestion datasets from two different cities and show the superiority of TRTF for both tasks of missing value completion and multi-step forecasting under different experimental settings. For instance,TRTF outperforms LSM-RN by 24% and TRMF by 29%.Conclusion. We present in this paper TRTF, an algorithm for temporal regularized tensor decomposition. We show how the algorithm can be used for several traffic related tasks such as missing value completion and forecasting. The proposed algorithm incorporates both spa-tial and temporal properties into the tensor decomposition procedures such as CP-WOPT, yielding to learning better factors. We also, extend TRTF with an auto-regressive procedure to allow for multi step-ahead forecasting of future values. We compare our method to recently developed algorithms that deal with the same type of problems using regularized matrix factorization,and show that under many circumstances, TRTF does provide better results. This is particularly true in cases where the data suffers from high proportions of missing values, which is common in the traffic context. For instance, TRTF achieves a 20% gain in MAPE score compared to the second best algorithm (CP-WOPT) in completing missing values in the case of extreme sparsity observed in Doha. As future work, we will first focus on adding non-negativity constraints to TRTF, although the highest fraction of negative values generated by our method throughout all the experiments did not exceed 0.7%. Our second focus will be to optimize TRTF training phase in order to increase its scalability to handle large dense tensors, and to implement it on a parallel environment.