Using Artificial Neural Networks for Equalizing Time Series Considering Seasonal Fluctuations

The objective of this contribution is to prepare a methodology of using artificial neural networks for equalizing time series when considering seasonal fluctuations on the example of the Czech Republic import from the People ́s Republic of China. If we focus on the relation of neural networks and time series, it is possible to state that both the purpose of time series themselves and the nature of all the data are what matters. The purpose of neural networks is to record the process of time series and to forecast individual data points in the best possible way. From the discussion part it follows that adding other variables significantly improves the quality of the equalized time series. Not only the performance of the networks is very high, but the individual MLP networks are also able to capture the seasonal fluctuations in the development of the monitored variable, which is the CR import from the PRC.


Introduction
Time series forecasting is currently one of the most significant fields of statistics in terms of development [1]. The main reason for the growing influence of this discipline is mainly the fact that it is capable of dealing successfully with the description of dynamic systems, which are often faced in practice [2]. The application of time series analysis occurs in every field of human activity, e.g. physics, technology, medicine, social sciences, and economics as well. Using time series is one of the most significant quantitative methods used in data analysis in economics [3].
Analysis of time series is important especially for the planning and forecasting crucial decisions in specific fields. Moreover, he claims that specifically, it is the field of operative management (forecasting the quantity of production), finance (forecasting the investment income), industry, economy (forecasting significant economic indicators) [4].
It is possible to forecast the indicators of foreign trade, i.e. export and import, based on time series as well [5]. Considering the subject of this paper, i.e. forecasting the Czech Republic import from the People's Republic of China, it is appropriate to examine the terms of the import and foreign trade of both countries more closely. Import is defined by the value of goods which has been received from abroad or crossed the frontier for limited or permanent stay in the country. It means that it involves the goods for consumption, cultivation, repair or re-export [6]. Both the Czech Republic and the People's Republic of China are open economies [7]. China is the biggest exporter and the second biggest importer for the Czech Republic, the ratios are 12.4% and 10.3% [8]. Chinese import into the Czech Republic is of larger volume, the structure of which is concentrated on investment goods and on such products that are further used for subsequent production [9]. Were recognize the groups of SITC 6 -market products, SITC 7 -machines and transport equipment and SITC 8 -industrial products as parts of the prevailing tradable groups [10].
It is typical for our times that information is recorded in the form of chronologically structured data, i.e. time series [2]. Line of time series was characterized as a sequence of substantially and spatially observations, which are timely structured from the past to the present. As mentioned above, time series is used in various disciplines, such as economics, medicine, physics or even meteorology [1]. There is an effort to define the development of monitored indicators in the past, find out the causes of variability and subsequently forecast the future in such disciplines [3].
If a seasonal component is the content of time series, it is necessary to clear it because it can obscure the dynamics of monitored phenomena and prohibit an objective comparison within the given year. The term of seasonal clearance can be understood as a removal of the seasonal component from the line of time series. In a model approach, the seasonal component represents a seasonal influence which is defined as a set of direct and indirect causes that reoccur annually. Concerning the analysis of seasonality, the first task is to define seasonal fluctuations in terms of their statistical significance. If the existence of seasonal component in the line of time series is proved, it is followed by the quantification of seasonal fluctuations [11].
If the data are seasonally cleared, it is possible to apply them for the construction of econometric models. There is a risk in the form of the reduced or distorted estimates of parameters even in this way. It is also very difficult to identify what the influence of seasonal clearing is on the accuracy of predictions. There is a general problem the essence of which is whether the line of time series is correctly seasonally cleared. It is possible to come across several criteria in practice which evaluate the quality of seasonal clearing. However, all of them are imperfect [12]. Prediction is a very demanding task in practice and there have been developed various methods for it. The methods on the basis of which it is possible to carry out the analysis and the prediction of time series and which take into account seasonal oscillations are for instance the decomposition of time series, seasonal differentiation, HP filter, a spectral analysis or Box-Jenkinson methodology [13].
Here has been an increased number of references to artificial neural networks (ANN) related to the analysis of time series and forecasting [14]. Artificial neural networks have been employed for prediction in the economy for decades [15]. Artificial neural networks are nowadays applied mainly for solutions to possible future problems [16]. The use of artificial neural networks is appropriate for complicated operations which we are not able to analytically identify. This is the reason why they are mostly used for the modelling of very demanding strategic decisions [17]. Neural networks are applied for classification, regression, an approximation of functions, understanding, identification of texts, generation of languages, etc. apart from dealing with time series [18][19]. Artificial neural networks differ from classical methods of by the ability to cope with a large quantity of data and the accuracy of results. Further advantages are their relatively easy use in predictions and complex problems [20]. Neural networks are very flexible in their use [21]. One of their disadvantages, is a requirement for the large quantity of data relating to a sample [22]. Consequently, it is necessary to carry out a lot of test observations, which is not very comfortable for users.
If we focus on the relation of neural networks and time series, it is possible to state that both the purpose of time series themselves and the nature of all the data are what matters. The purpose of neural networks is to record the process of time series and to forecast individual data points in the best possible way. It is necessary to direct neural networks in the right way so that they could deal with time series well [23].

Data and methods
The subject of analysis is the data on the Czech Republic import from the People´s Republic of China. The data is available on the websites of the World Bank, etc. For the analysis, the data on monthly balance starting from January 2000 and ending with July 2018 (in EUR) will be used, which is 223 input data. Table 1 shows descriptive characteristics of the input data set. An important phenomenon is the development of the Czech Republic import from the People´s Republic of China over time. The input data histogram and other selected statistical characteristics are presented in Figure 1 in graphical form.
The histogram corresponds to normal distribution. The data will be processed using DELL´s Statistica software, version 12. Regression using artificial neural networks will be carried out, when multilayer perceptron networks (MLP) and radial basis function (RBF) will be generated. There will be two sets of artificial neural networks generated: Dependent variable = the Czech Republic import from the People´s Republic of China, independent variable = time. Dependent variable = the Czech Republic import from the People´s Republic of China, continuous independent variable = time, categorical variable in the form of the month in which the value was measured = seasonal fluctuation. The process of work will be the same for both data sets. Time series will be divided into three sets -training, testing and validation data sets. Based on the training data set, which will contain 70 % of the input data, neural networks will be generated. Testing and validation data sets, both containing 15 % of the input data, will be used to verify the reliability of the neural network. A total of 10,000 neural networks will be generated, out of which 5 with the best characteristics will be retained. To determine the reliability of the neural network, the least squares method will be used. Generating of neural networks will be finished when there is no reduction of the value of the sum of the squares.
On the neural networks, whose sum of squares to the actual development of the Czech Republic import from the People´s Republic of China will be zero in ideal case, or at least as low as possible. The hidden layer of the MLP networks will contain at least 2, and no more than 50 neurons. The RBF network hidden layer will contain at least 21, but no more than 30 neurons. The delay of the time series will be 1. For the hidden and output layers, 5 distribution functions will be considered (linear, logistic, hyperbolic tangent, exponential, and sinus). Other settings will remain default. Finally, the results of both retained neural networks will be compared, and the findings from the research will be formulated.

Neural structures 1
Based on the methodology used, 10,000 neural networks were generated, out of which 5 with the best parameters were retained (see Table 2). It clearly results from the table that all 5 retained networks are RBF networks. The input layer contains one variable only -time. The output layer also contains only one variable, which is the Czech Republic import from the People´s Republic of China. The hidden layers contain 23 neurons (3 of the retained networks) and 29 neurons (2 out of the five retained networks). All networks used the RBFT training algorithm. The activation function for the hidden layer was Gaussian function, while the output activation function was Identity for all networks. Training, testing and validation performance of all retained networks is very interesting. Generally, it is necessary to find a network whose performance is ideally the same in all data sets. The error should be as small as possible (here, it shall be noted that the distribution of the data into the data sets was random). Table 3 shows the correlation coefficients of the training, testing and validation data sets, that is, the values of the performance of networks in these data sets. At first sight (in terms of correlation coefficient), the 2nd retained network, RBF 1-29-1, may appear to be the most suitable. However, it is necessary to carry out a more detailed analysis of the results obtained, and to give the basic statistical characteristics of the individual data sets for all retained neural networks (see Table 4). Source: authors.
In ideal case, individual statistics of the neural networks shall be identical in all data sets, both in terms of minimal and maximal forecasts, minimal and maximal residuals, and maximal standard residuals. In the case of equalized time series, the differences are minimal; however, it is not possible to identify which of the retained structures shows the best results. It is necessary to create a graphical representation of the actual development of the Czech Republic import from the People´s Republic of China, as well as the development of the forecasts of the individual retained networks (see Figure 2). The figure shows that all five retained networks´ forecasts of the development of the Czech Republic import from the People´s Republic of China are very similar. The shape of the balanced time series is also very similar to the actual import development; all networks thus appear to be very interesting. All five networks forecast the basic direction of the development of the Czech Republic import from the People´s Republic very well. These networks are also able to forecast local minimum and maximum. In terms of practical applicability, all networks appear to be applicable. The figure also confirms that the 2nd retained network (RBF 1-29-1) shows the best performance (however, the differences between the retained networks are minimal).

Neural structures 2
In the second case, there were generated 10,000 neural networks. Again, five of them with the best parameters were retained. The overview of the retained networks can be seen in Table 5. It results from the table that the retained networks are only multilayer perceptron networks. The radial basis function networks did not show characteristics suitable for being retained for the best performance. The input layer is characterized by two variables, one of them being time as a continuous variable, characterized by one neuron in the input layer. The second variable is the month of the measuring, which is categorical variable characterized by twelve neurons in the input layer. The hidden layers of the retained networks contain 3 -9 neurons. The output layer is logically characterized by one neuron and one output variable, which is the import (from the People´s Republic of China to the Czech Republic). The training algorithm used for all generated and retained networks was the Quasi-Newton algorithm, but used in three alternatives. The error function for all networks was the sum of the least squares. For three retained structures, the activation of the hidden layer was used by means of the Identity function, for two networks, it was the sinus function. The output activation function was the logistic function for three of the networks, in the case of the first retained network the exponential function was used, while for the fourth retained network, it was the Identity function. The correlation coefficients of the training, testing and validation data sets, that is, the performance values of the networks in these data sets are shown in Table 6. It shows that the performance values of all retained neural networks are quite high and also approximately the same. The small differences detected do not have any significant impact on the performance of the retained networks. The correlation coefficients for the training data set are above 0.90, while the 5th retained network, MLP 13-13-1, showed the highest value. For the testing data set, the correlation coefficients values are even higher, between 0.95-0.96, where the highest coefficient value is in the case of the 4th retained network, MLP 13-9-1.
The retained networks in the case of validation data set show the value of the correlation coefficient of 0.91-0.92, where the highest value is shown by the 4th retained network, MLP 13-9-1. At first sight, this network might appear to be the most suitable one. However, it is necessary to carry out a more detailed analysis of the results obtained, and to give basic statistical characteristics of the individual data sets for all retained neural networks (see Table 7). In ideal case, the individual statistics of the neural networks shall be horizontally the sane in all data sets in terms of minimal and maximal forecasts, minimal and maximal residuals, and minimal and maximal standard residuals. In the case of equalized time series, the differences are minimal, but it is not possible to determine unequivocally, which of the retained networks show the best characteristics. However, the 4th retained network MLP 13-9-1 appear to show very good results.
It appears to be necessary to make a graphical representation of the actual development of the Czech Republic import from the People´s Republic of China, as well as the development of forecasts of the individual retained networks (see Figure 3). The figure indicates that all retained neural networks´ forecasts of the development of the Czech Republic import from the People´s Republic of China are very similar. The curve of the equalized time series is also very similar to the actual development of the import; therefore, all retained networks appear to be very interesting. All 5 networks forecast the basic development trend of the Czech Republic import from the People´s Republic of China very well. Moreover, these networks are able to make good predictions of local minimum and maximum. In terms of practical applicability, all networks appear to be applicable. The figure also confirms that the 4th retained network, MLP 13-9-1, show the best results (nevertheless, the differences between the retained networks are minimal).

Comparison of A and B results
All generated and retained neural networks show very high performance in equalizing time series. The correlation coefficients of both RBF and MLP networks are higher than 0.9, which indicates a very high performance. However, assessing the performance only according to the correlation coefficients, it could be stated that the RBF networks show significantly better results than the MLP networks, where the sensitivity to seasonal fluctuations has been tested. It could be stated that the difference of max. 0.05 from the correlation coefficient value is not big, and does not play any role in the interpretation of the results. This is true to certain extent. However, comparing the graphical representation of the equalized time series with the actual course of the CR import from the PRC, it is clear at first sight that in particular the MLP networks are able to capture the seasonal fluctuations in the export, and the equalized time series successfully follow the past course of the monitored variable.
However, the correlation coefficients of training and validation data sets achieve lower values than those of the RBF networks. This can be mainly due to improper division of the data set into training, testing and validation data sets. This was done separately for each part of the experiment, that is, separately for the generation of the A and B networks results. The data were selected by means of the function randomsample. Although it is less likely, it is possible that an error occurred and the subsets are not equalized and cannot adequately represent the original data set. Another possible reason can be the calculation of the correlation coefficient, which works with mean values. These mean values can be better in the case of the RBF (although the results are not better in general). The position of the equalized time series compared to the actual course of the variable can thus be "advantageous" but the result can be inaccurate. Therefore, the conclusion should be that the time series equalized by means of MLP show much better results, with the 4th retained network, MLP 13-9-1 shows the best performance in general.

Conclusion
The objective of the contribution was to prepare a methodology of using artificial neural networks in equalizing time series and considering seasonal fluctuations on the example of the Czech Republic import from the People´s Republic of China. From the discussion part it follows that adding other variables significantly improves the quality of the equalized time series. Not only the performance of the networks is very high, but the individual MLP networks are also able to capture the seasonal fluctuations in the development of the monitored variable, which is the CR import from the PRC. Time series are relatively accurate in their course, and there are only slight fluctuations. This indicates possible applicability of MLP for equalizing time series affected with seasonal fluctuations.
This was achieved by adding categorical variable, which was intended to detect possible seasonal fluctuation. Moreover, it can be generalized that the seasonal fluctuations can be monitored in multiannual cycles, in a one-year cycle (monthly, quarterly), monthly cycles, weekly cycles. It always depends on what variable is being monitored. In the case of the CR import from the PRC, mainly the fluctuations in the individual months in a year were monitored. It was proved that the variable "month" enabled to achieve better results. There was also generated a neural network which is able to forecast the monitored period better than the other networks. It was the 4th retained network, MLP 13-9-1. As for the forecast itself, it is necessary to carry out an experiment with the time series delay that could bring interesting results in this case. The objective of the contribution was achieved.