Redirigiendo al acceso original de articulo en 18 segundos...
ARTÍCULO
TITULO

Time series forecasting in real-time streaming data processing

R.A. Elchenkov    
M.E. Dunaev    
K.S. Zaytsev    

Resumen

The purpose of this work is to study methods for predicting the values of time series when processing streaming data in distributed systems in real time. To do this, the authors propose a modification of the autoregressive model with a given AR order by adding to it the inheritance function of the previous values of the time series. The results of comparative experiments of the proposed modification, called Real-Time AR with classical AR and ARIMA, confirmed the effectiveness of the modification. This is especially evident in the presence of anomalies in the behavior of the real time series. The proposed modification of the algorithm allows not only to parallelize calculations, but also to configure the model on the fly in the Apache Spark ecosystem. To conduct experiments with the algorithms, a special data array was built - a data slice from 1000 measurements of the Apache Kafka server metrics log with one topic, two producers and one consumer. Anomalous fragments were artificially added to the array, differing in a large number of messages per second and/or message size. The values of the proposed data array were normalized and shifted by the average value over the training sample of the model pre-training. The results of applying the proposed algorithm in solving problems of predicting the values of time series showed that the presence of anomalies in the behavior of objects does not introduce significant distortions in the results of predicting values.

 Artículos similares

       
 
Yong Zhang, Xin Wang, Zongli Jiang, Junfeng Wei, Hiroyuki Enomoto and Tetsuo Ohata    
Arctic glaciers comprise a small fraction of the world?s land ice area, but their ongoing mass loss currently represents a large cryospheric contribution to the sea level rise. In the Suntar-Khayata Mountains (SKMs) of northeastern Siberia, in situ measu... ver más
Revista: Water

 
Jianzhao Liu, Liping Gao, Fenghui Yuan, Yuedong Guo and Xiaofeng Xu    
Soil water shortage is a critical issue for the Southwest US (SWUS), the typical arid region that has experienced severe droughts over the past decades, primarily caused by climate change. However, it is still not quantitatively understood how soil water... ver más
Revista: Water

 
Angel E. Muñoz-Zavala, Jorge E. Macías-Díaz, Daniel Alba-Cuéllar and José A. Guerrero-Díaz-de-León    
This paper reviews the application of artificial neural network (ANN) models to time series prediction tasks. We begin by briefly introducing some basic concepts and terms related to time series analysis, and by outlining some of the most popular ANN arc... ver más
Revista: Algorithms

 
Yi Ouyang, Tao Feng, Han Feng, Xinghan Wang, Huayu Zhang and Xiaoxue Zhou    
Deformation monitoring plays a pivotal role in assessing dam safety. Interferometric Synthetic Aperture Radar (InSAR) has the advantage of obtaining an extensive range of deformation, regardless of weather conditions. The Datengxia Water Conservancy Hub ... ver más
Revista: Water

 
Altayeb A. Obaid, Elhadi M. Adam, K. Adem Ali and Tamiru A. Abiye    
The Vaal Dam catchment, which is the source of potable water for Gauteng province, is characterized by diverse human activities, and the dam encounters significant nutrient input from multiple sources within its catchment. As a result, there has been a r... ver más
Revista: Water