Early Recognition of Rainfall to Plan and Manage the Cultivation: A Forecasting Model Using Artificial Neural Networks for Dompe Divisional Secretariat in Gampaha District, Sri Lanka

As the agricultural productivity is climate sensitive, forecasting climatic factors is important to maximize the harvest and to planning and management of the cultivation. Even though all the climatic factors have influence on cultivation, rainfall is one of the major influential factor to Sri Lankan agriculture. Rainfall forecasting is vital in agriculture and it is a challenging task due to the uncertainty of natural phenomena. Artificial neural network (ANN) approach is being applied for forecasting real time rainfall using climatic factors affecting to rainfall. The aim of this study was to identify a neural network model which is capable of forecasting rainfall of Dompe Divisional Secretariat in Gampaha District with a high accuracy. Feed forward neural network model (FFNN) was applied with Levenberg-Marquardt back propagation algorithm and the network parameters were adjusted to minimize the forecasting error. The suggested FFNN model consisted of 12 input variables with two hidden layers with 13 and 10 hidden neurons in first and second layers, respectively. Log sigmoid transfer function used in hidden layer 1 and 2 while pure linear transfer function was used in the output layer. The model forecasts rainfall with mean squared error of 0.0895 and normalized mean squared error of 0.1975. The coefficient of determination (R2) of the testing set was 0.8. These results demonstrated the suitability of other climatic factors: temperature, wind speed, air pressure, humidity, percentage of clouds and their lags in forecasting rainfall using ANN technique in forecasting rainfall with high accuracy.


INTRODUCTION
Climate change is any change in climate over time, whether due to natural variability or as a result of human activity (Praveen and Sharma, 2020). Climate change may cause risks at cultivation which may be unavoidable and unexpected. The possible effects of climate changes on agricultural production are not only limited to the crop cultivation but also cause fluctuations on quantity of the crop yields. Plant growth and development are regulated by climatic conditions (Adams et al., 1990). According to past studies, some of the most important climatic factors that would affect agricultural productivity are rainfall (Neenu et al., 2013), changes in temperature (Hatfield and Prueger, 2015), carbon dioxide fertilization, short term weather variability and surface water run off (Joshi and Amalkar, 2009). A crop cultivation is highly influenced by climatic factors like temperature of the earth surface and in atmosphere, groundwater level, flooding due to high rainfall, drought, soil erosion, heavy wind, rising sea level due to melting of glacier, cyclone, wind speed, hail storm, fog, earthquake and landslide (Praveen and Sharma, 2020). Agricultural productivity also is climate sensitive (Neenu et al., 2013), and can be influenced by the fluctuations in temperatures and rainfall pattern. Adverse effects of climate change on the productivity of food grain crops may threaten food security (Praveen and Sharma, 2020). The crop water regime may be affected by changes in seasonal precipitation, within-season pattern of precipitation, and inter annual variation of precipitation (Neenu et al., 2013). Forecasting of climatic factors is therefore required to optimize the quantity of the yield that can be achieved under different climate change scenarios.
Among climatic factors, rainfall is one of the major influential factors in Sri Lankan agriculture. Rainfall and its variability influence all sectors of the agriculture in several ways. Rainfall variation adversely affect the quantity of the harvest. The rate of change of rainfall is fluctuated with the time. As it is changing with time, it is important to have an accurate rainfall forecasting model. Rainfall prediction is very important for decision makers in agriculture. Accurate information on rainfall is essential for the planning and management of cultivation (Praveen and Sharma, 2020). A process of rainfall forecasting is valuable only if it is useful to make decisions that yield attractive benefits to farmers (Hung et al., 2009;Katz and Murphy, 1997).
Rainfall is affected by the temperature, humidity, wind speed, air pressure, dew point (Shaikh et al., 2017). Further, Hung and Babel (2009) observed that relative humidity, air pressure, wet bulb temperature, cloudiness are the most important climatic factors which should be considered in rainfall forecasting. Based on these parameters, rainfall forecasting models can be developed using artificial neural networks. Artificial neural networks (ANN) have been widely applied to modeling data as it is flexible and capable of nonlinear modeling (Nanayakkara et al., 2014). Shaikh et al. (2017) have observed that ANN has a very high potential to be used to forecast rainfall and it has better accuracy than statistical and mathematical models. On the other hand, ANN can be used for predictions as well because it has a capability of examining and determining the nonlinear behavior of historical data used for prediction (Kolarik and Rudorfer, 1994).
Artificial neural network is a type of data driven technique which refers the principle of biological neurons (Seyyedabbasi et al., 2018). This was firstly introduced by McCulloch andPitts in 1943 (MCCulloch et al.,1943). Hu (1964) implemented ADALINE system for weather forecasting, which became one of the earliest applications of ANN in weather forecasting. The most common neural networks used in predictions in past studies is the feed forward neural network. Back propagation is a widely used algorithm for training feed forward neural network (Shaikh et al., 2017).
Western Province is a highly populated province in Sri Lanka with rapid urbanization with a land extent of 367,312 ha. Even though Gampaha District, one of the three Districts in the Western Province, is urbanizing rapidly, cultivation still remains as major livelihood in rural sectors of this district. Therefore, it is important to propose an accurate rainfall forecasting model to the Gampaha District. Therefore, this study aims at modelling and forecasting of rainfall in a selected study area located in the Gampaha District.

METHODOLOGY
All theories and methods used throughout the study and data used in the study are described in this section.

Artificial neural network
The ANN is a computer program which is designed similar to natural neural network or human brain where some cells are responsible for receiving the external stimulus, some for processing and some for the transfer the response to the intended part. Neural network is made up of an input layer, one or more hidden layers and an output layer. All layers | 193 are interconnected by means of connectors of different weights. Similarities of human brain and ANN can be introduced as follows: Neuron -Cell/ Node, Synapse -Connections/Links, synaptic efficiency -weight/strength of the connection, firing frequency -output node. Hence, components of an artificial neural network are input, output, weights, bias and activation function.

Multilayer neural networks
A multi-layer neural network contains more than one layer of artificial neurons or nodes. Different number of hidden neurons and different types of transfer functions can be used in layers (Shaikh et al., 2017). Multilayer neural networks are used in estimating nonlinear patterns.

Feed forward neural networks
Feed forward neural network is a type of an artificial neural networks where the connections between units do not form a cycle, that is information flows only in forward direction.

Training an artificial neural network
Once a network has been structured for a particular application, that network is ready to be trained. To start this process, the initial weights are chosen randomly. Then, the training, or learning, begins. There are two approaches for training: -Supervised learning and Unsupervised learning. In supervised learning, both the inputs and the outputs are provided. The network then processes the inputs and compares its resulting outputs against the desired outputs. Errors are then propagated back through the system, causing the system to adjust the weights which control the network. In unsupervised training, the network is provided with inputs but not with desired outputs (Abhishek, 2012).
Process of training ANN is the most important task in developing a neural network model. To obtain the best performance of ANN, following features will be changed one at a time: input combination, number of hidden layers, number of hidden neurons in each layer, transfer function in each layer, training algorithm, combination coefficient, increase/ decrease factor of combination coefficient.

Back propagation algorithm
The Backpropagation algorithm is a supervised learning method for multilayer feed-forward networks. The principle of the backpropagation approach is to model a given function by modifying internal weightings of input signals to produce an expected output signal. The system is trained using a supervised learning method, where the error between the system's output and a known expected output is presented to the system and used to modify its internal state. Technically, the backpropagation algorithm is a method for training the weights in a multilayer feed-forward neural network (Hung et al., 2009).
Several backpropagation algorithms are available and Levenberg Marquardt back propagation algorithm is one of them where weights and biases updated according to Levenberg Marquardt optimization. It is a combination of gradient decent algorithm and Gauss Newton method. This algorithm designed to approach second order training speed without having to compute hessian matrix.

Data
The data set contains monthly records of rainfall (mm), temperature (°C), wind speed (kmph), air pressure (mb), humidity (%) and percentage of clouds form January 2010 to December 2019. Data set was divided into three parts and used for training (80%), validation (10%) and testing (10%) before apply to the model (Shaikh et al., 2017). The training data set was used for weight and bias updating process. A practical way to find an optimal point of better generalization and avoid over fitting is to set aside a small percentage of the training data set named as validation data set. Performance of the models were evaluated using the test set.

Performance measures
Performance of the suggested models were measured based on the forecasting error. That is the deviation of forecasted values from the actual value. As formal measure, the means squared error (MSE) and the normalized mean squared error (NMSE) were used as the performance indicator in this study.

Model development
As this study aims at forecasting rainfall using other climatic factors, correlation analysis was carried out between rainfall and other climatic factors using both Pearson's and Spearman Rho correlation coefficients. Then cross correlations, serial correlations and partial auto correlations of those variables were identified. Based on Pearson's correlation coefficient, Spearman Rho coefficient, serial correlation coefficient, cross correlation, ACF and PACF, determined the input combination to the neural network as follows: Temperature (TM), Wind speed (WS), Pressure (PS), Cloud percentage (CD), Humidity (HD), Temparature_lag1 (TL1), Windspeed_lag1 (WSL1), Windspeed_lag2 (WSL2), Pressure_Lag1(PSL1), Pressure_Lag2 (PSL2), Pressure_Lag3 (PSL3), Humidity_Lag1 (HDL1), cloud_Lag1 (CDL1). Output variable is Rainfall. TL1 represents the temperature of one month before the observed period. WSL2 represents the wind speed of two months before the observed period. Similarly, this applies to other aforementioned variables.
Feed forward neural networks were developed and trained for climatic data in order to obtain a rainfall forecasting model. Prior to using, the data set was normalized as there were different scales in different variables. To obtain a most suitable model, different parameters of the ANN were changed one at a time while keeping other parameters constant. Following parameters were changed in different iterations: input combination, number of hidden layers, number of hidden neurons in each layer, transfer function in each layer, training algorithm and parameters of the training algorithm (Abhishek, 2012). Network was trained for thousands of times to find out the most appropriate model. The first step was the identification of the most suitable input combination. The variables were included or excluded based on the correlation coefficient between independent variables and dependent variable. The variables with least correlation with rainfall were removed one by one and observed the changes in the performance of the model. Out of those trials minimum NMSE yield the most suitable combination of inputs.
Second step was to train the model by changing number of hidden layers. One hidden layer with one neuron was used at the beginning. Then increased the number of hidden layers one by one and changed the number of neurons in each layer until the minimum error was obtained (Hung et al., 2009). Log sigmoid, pure linear and tan sigmoid transfer functions were used with different combinations of hidden neurons in hidden layers. The neural network was trained with Levenberg Marquardt back propagation algorithm and identified the optimal combination coefficient. The number of iterations in the training process is known as 'Epochs'. The maximum number of epochs used in this study was 1000. The network performance was measured by MSE and NMSE.

RESULTS AND DISCUSSION
This section describes relationships between variables and performance of the neural network in each trial under training process until reach the optimal model. Figure 1 illustrates the scatter plots to represent the relationship between rainfall and other climatic factors: temperature, wind speed, air pressure, humidity, cloud percentage. Table 1 represents the Person's and Spearman Rho correlation between rainfall and other selected climatic factors and its lags. These correlation coefficients were used to identify the input combination for the model.

Feed forward neural network for climatic data
The objective of this study is to forecast rainfall using significant climatic factors as it is one of the major influential factor for the cultivation. The challenge was to collect meteorological data of all climatic factors for particular areas where cultivation is processing in large scale. Therefore, this study considered the records of climatic factors in Dompe Divisional Secretariat in Gampaha District. Climatic data set include 1560 observations of 13 variables including lags of the variables. First 80% of the data set which contained 1248 observations were used to train the network.

Parameter
Correlation

Combination of input
There are 13 variables in the data set as inputs which resulted 8192 possible combinations. Practically it is difficult to test all these combinations one by one. Therefore, this study was carried out using only few most important combinations based on correlation coefficients which indicate the relationship between variables.
Then variables were removed one by one and observed the MSE and NMSE as given Table 2.
According to the results in the Table 2, the combination with 12 inputs has the minimum error. Hence, this input set was used for further improvements of the model.

Number of hidden layers
Number of hidden layers were changed from one to four for the selected model. According to the results (Table 3), minimum NMSE is recorded under two hidden layers. But minimum MSE was recorded under one hidden layer. Thus, it was decided to proceed the training of the neural network with one and two hidden layers separately

Scatterplots to represent the relationship between rainfall and other climatic factors
.

Number of hidden neurons in hidden layers
For the single hidden layer, the most suitable number were identified by changing the number of neurons from 1 to 20. Then considered two hidden layers. and the most suitable number of neurons in each hidden layer were identified by changing the number of neurons in the first hidden layer from 1 to 20 while changing the number of neurons in second layer from 1 to 20 simultaneously.
The data set was rained for several trials and observed that 13 neurons in the hidden layer captured the minimum error (Table 4). Then trained and observed the results of the neural network with two hidden layers. It was observed that 13 neurons in first hidden layer and 10 neurons in the second hidden layer captured the  minimum error (Table 5). Thus, it was decided to proceed the training of the neural network with 13 and 10 neurons in first and second hidden layers, respectively.

Transfer function in each layer
There are many transfer functions used in neural networks. Log-sigmoid, tan sigmoid and pure linear are most popular functions among them.
It was identified that the transfer function logsigmoid was suitable for hidden layer one and two wile the pure linear function was appropriate for output layer (Table 6). Training algorithm There are several backpropagation training algorithms available which are used to train the neural networks. Out of these training algorithms Levenberg-Marquardt back-propagation algorithm yielded the minimum NMS and NMSE (Table 7).

Change the parameters of the algorithm
After selecting the training algorithm, the model was trained by changing model parameter: combination coefficient (µ) until the minimum error was obtained. The default value of µ was 0.001 and it was changed from 0.00001 up to 0.01 The combination coefficient 0.0001 for Levenberg-Marquardt back-propagation algorithm yield the minimum error (Table 8). Therefore, suitable value of µ for this model was decided as 0.0001. With those adjustments the model improving process.

Final model
After thousands of trials, the best model was selected based on the MSE and NMSE of the test data set. The final model includes 12 variables: TM, WS, PS, HD, CD, TL1, WS1, PSL1, PSL2, PSL3, HDL1, CDL1. There are two hidden layers with 13 and 10 hidden neurons in first and second layer, respectively. Log sigmoid transfer function used in hidden layer 1 and 2 while a pure linear transfer function was used in the output layer as transfer  Figure 2 illustrates the architecture of the final model.
Actual data used for training, validation and testing were overlayed with the model predicted data for the studied period ( Figure 3). For better illustration the above comparison was done separately only for the testing data set (Figure 4). We observed that the FFNN model developed in this study was capable of forecasting rainfall with a MSE of 0.0895 and NMSE of 0.1975.   Figure 4, it can be identified that the final feed forward neural network model is capable of forecasting rainfall with higher accuracy. This model requires climatic records of current month to forecast rainfall for next month and future. Early recognition of distress from rainfall on cultivation before one month with the developed model in this study will leads to maximize the quantity of the yield of cultivation.
Even though this study focus on agrarian lands of Dompe Divisional Secretariat in Gampaha District, rainfall data was collected from only one station. This is the major limitation of this study. Further, there are several other factors that influence on rainfall such as dew point (temperature to which air must be cooled to become saturated with water vapor) and CO2 concentration in atmosphere. Those data do not available for the study area for the relevant period.
This study focuses on feedforward neural network as the forecasting model and in future studies, other types of neural networks and other data mining techniques can be used. Further, the same technique can be used to forecast other climatic factors affecting cultivation which leads to have better cultivation plan.

CONCLUSIONS
Identified feed forward neural network model forecasts rainfall with a MSE of 0.0895 and NMSE of 0.1975 in Dompe Divisional Secretariat in Gampaha District.