Stress Classification using Deep Learning with 1D Convolutional Neural Networks

ABSTRACT

Stress has been a major problem impacting people in various ways, and it gets serious every day. Identifying whether someone is suffering from stress is crucial before it becomes a severe illness. Artificial Intelligence (AI) interprets external data, learns from such data, and uses the learning to achieve specific goals and tasks. Deep Learning (DL) has created an impact in the field of Artificial Intelligence as it can perform tasks with high accuracy. Therefore, the primary purpose of this paper is to evaluate the performance of 1D Convolutional Neural Networks (1D CNNs) for stress classification. A Psychophysiological stress (PS) dataset is utilized in this paper. The PS dataset consists of twelve features obtained from the expert. The 1D CNNs are trained and tested using 10-fold cross-validation using the PS dataset. The algorithm performance is evaluated based on accuracy and loss matrices. The 1D CNNs outputs 99.7% in stress classification, which outperforms the Backpropagation (BP), only 65.57% in stress classification. Therefore, the findings yield a promising outcome that the 1D CNNs effectively classify stress compared to BP. Further explanation is provided in this paper to prove the efficiency of 1D CNN for the classification of stress. without human intervention [13]. Machine learning provides the ability for machines to train on and accordingly adapt when there is new data input, enables the prediction from known data, and learns from the new unknown data.
One of the machine learning methods that can be used to recognize stress is Artificial Neural Networks (ANNs). ANNs are a non-linear data-driven self-adaptive approach instead of the traditional model-based methods" by Solomatine & Ostfeld [14]. "ANNs can determine and study correlated patterns between input data and corresponding target values. It can use to predict the outcome of new independent input data after training. Neural networks (NNs) "have been used in a wide variety of applications where statistical methods are traditionally employed like classifying problems or predicting outcomes as said by Solomatine & Ostfeld [14].
For a very long period, ANNs had to be limited in size due to difficulties in training. However, Hinton, Osindero, and Teh [15] suggested a procedure that could be used with much larger networks known as Deep Learning (DL). LeCun, Bengio, and Hinton [16] said Deep learning allows computational models composed of multiple processing layers to learn the representation of data with multiple levels of abstraction. Convolutional Neural Networks (CNNs), one of the most well-known DL models, have become the de facto standard for many Computer Vision and Machine Learning operations over the last decade. CNNs are feed-forward Artificial Neural Networks (ANNs) with convolutional and subsampling layers that alternate. When trained on a large visual database with ground-truth labels, deep 2D CNNs with many hidden layers and millions of parameters can learn complex objects and patterns. With proper training, this one-of-a-kind ability transforms them into the primary tool for various engineering applications involving 2D signals such as images and video frames. As an alternative, 1D Convolutional Neural Networks (1D CNNs) have recently been developed as modified 2D CNNs. 1D CNNs have been utilized in several classification problems like heart sound and Soil Texture [17][18] [19]. It is believed that the classification between stress and nonstress will be more accurate with multiple processing layers compared to the ANNs that only consist of a single processing layer. Smets et al. [20] said the previous approach to measure stress does not allow for continuous monitoring and often suffers from biases such as demand effects, response, and memory biases; therefore, the focus has been shifted towards measuring bodily responses as indicators of stress.
This study aims to use stress datasets to anticipate an accurate output; society will use the outcomes. The remaining parts of this paper are organized as follows: methodology and empirical studies are presented in section 2; Findings and the analysis of the results are presented in sections 3 and 4, and finally, section 5 concludes the paper's future works.

II. Methods
This section introduces the research methodology that had been used in this research. The components in this section include Dataset Preparation, Data pre-processing, Research Design, Research Implementation, and Instruments, as in Figure 1. DL has become a growing research interest, and the method has shown certain advantages of learning performance. DL can learn from the past data to solve complex problems and has been widely used in classification. It allows the computational models with various processing layers to learn the data representation with various levels of abstraction. The result of the DL method is compared to one of the neural network standards, the Backpropagation (BP) algorithm, which was used in previous studies related to stress classification [21] [22]. This comparison is to analyze the effectiveness of the DL algorithms in this study. The research focuses on psychophysiological stress, which looks at heart rate variability (HRV), galvanic skin response (GSR), skin temperature (Temp), and respiration.

A. 1D Convolutional Neural Networks (1D CNNs)
1D CNNs is very similar to CNNs where the input of 1D CNNs is dimensional, whereas an ordinary CNNs is two-dimensional. A Convolutional Neural Network (CNN) is very similar to ordinary neural networks as they are made up of neurons that have learnable weight and biases. Each neuron receives some inputs, performs a dot product, and optionally follows it with a non-linearity. Loss function still can be found in the last layer or fully connected layer, and all the tips developed for learning regular neural networks still apply. Figure 1 shows the schematic representation of 1D CNNs. The first hidden layer is always a convolutional layer, followed by a pooling layer. The convolutional and pooling layers can be present multiple times in the network until the final hidden layer is the full-connected layer. The Schematic Representation of 1D Convolutional Neural Networks is as shown in Figure 2.
The convolution layer is where the previous layer's feature maps are folded with learnable kernels and execute the activation function to form the output feature map. Each output map may combine convolutions with multiple input maps. The mathematical formula for constructing the convolution layer is used (1).
where is the selection of input maps, each output map is given an additive bias , for an output map, the input maps will be folded with different kernels. So, if output map and map both sums over input map , the kernels applied to map are different for output maps and .
The pooling layer produces downsampled versions of the input maps. If there are N input maps, there will be exactly N output maps, although the output map is smaller (2).
where ( ℓ−1 ) represents a sub-sampling function. This function will sum over each distinct nby-n block in the input so that the output is n-times smaller along both spatial dimensions. Each output map is given its multiplicative bias and an additive bias .
The fully connected layer will combine features learned by different convolution kernels so that the network can build a global representation of the holistic input. Let Denote the weight given to input map when forming the output map . The output map is (3).
Subject to the constraints (4).
These constraints can be enforced by setting the variables equal to the softmax over a set of unconstrained, underlying weight use (5).
Each set of weights for fixed j are independent of all other such sets for any other , the single map is updated, and the subscript . Each map is updated in the same way except with different indices.

B. Empirical Studies
Several experiments are conducted on the psychophysiological stress dataset. The implementation is multivariate analysis to evaluate 1D CNNs deep learning model. There is a total of twelve features in the raw data given by Dr. Elena Smets [20], which include Skin Conductance / Galvanic Skin Response (SC/GSR), Skin Temperature (Temp), Blood Volume Pulse (BVP), Respiration Changes (RSP), Heart Rate (HR), Electromyography amplitude (EMG amp), Blood Volume Pulse amplitude (BVP amplitude), Respiration Changes rate (RSP-Rate), Respiration Changes amplitude (RSPamplitude), Heart Rate Variability amplitude (HRV amplitude), Respiration Changes + Heart Rate Coherence (RSP+HR Coherence), and Segments. The segments will display the state of the respondents at that moment. For each sensing modality, features have been calculated on a sliding window of 30 s with 29 s overlap. The illustration of the datasets is shown in Figure 3.
The dataset had been separated into two parts: a training set and a testing set. The training set usually consists of input vectors and corresponding output vectors that fit into the model to produce a result compared with the target for each input vector in the training dataset. The testing set is utilized Fig. 3. Psychophysiological stress dataset to provide an unbiased evaluation of a final model fit on the training dataset. The dataset had been normalized using the Z-Score normalization technique to scale the dataset value from 0 to 1. The normalized data then fits into the proposed deep learning model for classification. The classification performance of the deep learning model will be evaluated by utilizing the k-fold cross-validation. Brownlee [23] said that Cross-validation is a resampling procedure used to evaluate a limited data sample of machine learning models. In k-fold cross-validation, the dataset is divided into k subsets of equal size. The model is built fork times, leaving one of the subsets from training and using it as the test set.
Python is a hugely popular general-purpose programming language, so it is beneficial to apply solidly to AI and profound learning principles. Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano. TensorFlow serves as the backend engine for the Keras API to handle the low-level operations such as tensor products and convolutions. The machine used is Intel® Core™ i7-4720HQ CPU @ 2.60GHz 2.59GHz with 8.00GB RAM. The proposed method uses the 1D CNNs deep learning model to classify. 1D CNNs works precisely like normal CNNs but with different input dimension. The input for normal CNNs usually be two-dimensional or image datasets, but the input for 1D CNNs is one-dimensional or signal datasets. Table 1 shows the dataset, batch size, epochs, activation function, and optimizer used in this study.
The Batch Size used is 256. Batch Size is the number of training examples utilized in one iteration. The data was run for 15 epochs to see the model's performance. Epoch is one forward pass and one backward pass of all the training examples. The activation functions are ReLU and Softmax. ReLU or Rectified-Linear Unit Layer will provide a threshold at zero. Softmax function or normalized exponential function is used to represent categorical distribution. The optimizer function is SGD. SGD or Stochastic Gradient Descent is a simple yet efficient approach to discriminative learning of linear classifiers under convex loss functions such as Support Vector Machine or Linear Regression. Figure 4 shows the schematic representation of the 1D CNNs model.
The classification accuracy of the output is calculated according to the use of equation (6).   Table 2 shows the results of the 1D CNNs deep learning model with 10-fold cross-validation. In each fold, the data had been divided into ten groups. The first group will be used as testing in the first fold, and the other groups will be used as training. In the second fold, the second group will be used as testing, and the other groups will be used as training, followed by the third group for testing in the third fold. This will recursively continue until the 10 th fold is completed. Besides that, the plotting of accuracy and loss function for each training process and testing process also can be seen in Table 2. The Final accuracy for 1D CNNs is 99.97%.

III. Results and Discussions
The dataset is tested in a second experiment with The Backpropagation (BP) algorithm for further proof. BP is fit into the same data to compare the performance of both models. In Table 2, it is clearly stated that the training classification performance of 1D CNNs is 100% and the testing classification performance of 1D CNNs is 99.97%. However, the training classification performance of BP is 65.85%, and the testing classification performance of BP is 65.57%. Figure 5 clearly shows that the 1D CNNs deep learning model outperforms the BP algorithms. Moreover, our algorithm achieved higher accuracy than the methods mentioned in [24]. Table 3 compares the implementation of the CNN and other recent works based on different machine and deep learning algorithms. As shown in Table 3, our work achieves better accuracy than most of the other recent works (which have been reported in this paper) except that of [20], which achieved 99.8% using the physiological signals dataset. However, our work's accuracy gains better results than the same dataset.
The datasets must be structured appropriately to fit perfectly into the model and provide a good analysis of the results. Every step, from loading the dataset from a text file or CSV file to evaluating the trained model's performance, is critical because simple errors can cause errors when the code is run. The parameters used to build the model must be tested repeatedly to find the best parameter. For example, a sigmoid may be better when ReLU is a terrible choice and vice versa. Building the best model requires a lot of trial and error. The deep learning model is well-known for its high accuracy, regardless of whether it can solve a classification problem or a regression problem.  [25] Physiological signals 99.80% ANN [26] WESAD dataset 95.21% SVM [20] Psychophysiological Stress 93.4 ± 3.2% Convolutional Neural Network (CNN) [27] The heart rate signal 98.69 ± 0.45% Machine learning [28] The mind-brain-body dataset 81.33% Machine learning [29] Questionnaire (DASS 21) Between 0.628 -0.798 Hybrid deep learning technique [30] Employee Data 96.2 % This work Psychophysiological Stress 99.7%

IV. Conclusion
In this study, the 1D CNNs model was utilized to classify stress and compared with BP and various techniques from the Literature review. A psychophysiological stress dataset was used in this study. The performance of the developed model is evaluated by comparing it with the BP model and other works. The result obtained in this study shows that the 1D CNNs model is reliable in classifying stress with low lost function, which is 0.001, and high accuracy of 99.97%. More datasets are required to enhance the result of stress classification in the future. Skill and experience are essential in assisting the person in selecting and checking the best method in the modeling process. Spiking Neural Network as the latest generation of neural network and other areas of study will be applied to do stress classification.