Human Facial Expressions Identification using Convolutional Neural Network with VGG16 Architecture

Luther Alexander Latumakulita; Sandy Laurentius Lumintang; Deiby Tineke Salakia; Steven R. Sentinuwo; Alwin Melkie Sambul; Noorul Islam

doi:10.17977/um018v5i12022p78-86

Human Facial Expressions Identification using Convolutional Neural Network with VGG16 Architecture

Luther Alexander Latumakulita, Sandy Laurentius Lumintang, Deiby Tineke Salakia, Steven R. Sentinuwo, Alwin Melkie Sambul, Noorul Islam

Abstract

The human facial expression identification system is essential in developing human interaction and technology. The development of Artificial Intelligence for monitoring human emotions can be helpful in the workplace. Commonly, there are six basic human expressions, namely anger, disgust, fear, happiness, sadness, and surprise, that the system can identify. This study aims to create a facial expression identification system based on basic human expressions using the Convolutional Neural Network (CNN) with a 16-layer VGG architecture. Two thousand one hundred thirty-seven facial expression images were selected from the FER2013, JAFFE, and MUG datasets. By implementing image augmentation and setting up the network parameters to Epoch of 100, the learning rate of 0,0001, and applying in the 5Fold Cross Validation, this system shows performance with an average accuracy of 84%. Results show that the model is suitable for identifying the basic facial expressions of humans.

Full Text:

PDF

References

D. L. Z. Astuti, S. Samsuryadi, and D. P. Rini, “Real-Time Classification Of Facial Expressions Using A Principal Component Analysis And Convolutional Neural Network,” SINERGI, vol. 23, no. 3, p. 239, Oct. 2019.

L. F. Barrett, R. Adolphs, S. Marsella, A. M. Martinez, and S. D. Pollak, “Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements,” Psychol. Sci. Public Interes., vol. 20, no. 1, pp. 1–68, Jul. 2019.

Y. Park and M. Garcia, “Pedestrian safety perception and urban street settings,” Int. J. Sustain. Transp., vol. 14, no. 11, pp. 860–871, Sep. 2020.

[4] S. Simpson, L. Richardson, G. Pietrabissa, G. Castelnuovo, and C. Reid, “Videotherapy and therapeutic alliance in the age of COVID‐19,” Clin. Psychol. Psychother., vol. 28, no. 2, pp. 409–421, Mar. 2021.

D. T. Cordaro, R. Sun, D. Keltner, S. Kamble, N. Huddar, and G. McNeil, “Universals and cultural variations in 22 emotional expressions across five cultures.,” Emotion, vol. 18, no. 1, pp. 75–93, Feb. 2018.

A. J. Umaña‐Taylor and N. E. Hill, “Ethnic–Racial Socialization in the Family: A Decade’s Advance on Precursors and Outcomes,” J. Marriage Fam., vol. 82, no. 1, pp. 244–271, Feb. 2020.

S. M. Saleem Abdullah and A. M. Abdulazeez, “Facial Expression Recognition Based on Deep Learning Convolution Neural Network: A Review,” J. Soft Comput. Data Min., vol. 02, no. 01, Apr. 2021.

L. Zahara, P. Musa, E. Prasetyo Wibowo, I. Karim, and S. Bahri Musa, “The Facial Emotion Recognition (FER-2013) Dataset for Prediction System of Micro-Expressions Face Using the Convolutional Neural Network (CNN) Algorithm based Raspberry Pi,” in 2020 Fifth International Conference on Informatics and Computing (ICIC), Nov. 2020, pp. 1–9.

D. V. Sang, N. Van Dat, and D. P. Thuan, “Facial expression recognition using deep convolutional neural networks,” in 2017 9th International Conference on Knowledge and Systems Engineering (KSE), Oct. 2017, pp. 130–135.

Y. Gultom, A. M. Arymurthy, and R. J. Masikome, “Batik Classification using Deep Convolutional Network Transfer Learning,” J. Ilmu Komput. dan Inf., vol. 11, no. 2, p. 59, Jun. 2018.

S. Porcu, A. Floris, and L. Atzori, “Evaluation of Data Augmentation Techniques for Facial Expression Recognition Systems,” Electronics, vol. 9, no. 11, p. 1892, Nov. 2020.

A. Caroppo, A. Leone, and P. Siciliano, “Comparison Between Deep Learning Models and Traditional Machine Learning Approaches for Facial Expression Recognition in Ageing Adults,” J. Comput. Sci. Technol., vol. 35, no. 5, pp. 1127–1146, Oct. 2020.

C. Modarres, N. Astorga, E. L. Droguett, and V. Meruane, “Convolutional neural networks for automated damage recognition and damage type identification,” Struct. Control Heal. Monit., vol. 25, no. 10, p. e2230, Oct. 2018.

D. Keltner, D. Sauter, J. Tracy, and A. Cowen, “Emotional Expression: Advances in Basic Emotion Theory,” J. Nonverbal Behav., vol. 43, no. 2, pp. 133–160, Jun. 2019.

G. Ramirez-Gargallo, M. Garcia-Gasulla, and F. Mantovani, “TensorFlow on State-of-the-Art HPC Clusters: A Machine Learning use Case,” in 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID), May 2019, pp. 526–533.

S. A. Asmai*, M. N. D. Mohamad Zukhairin, A. S. M. Jaya, A. F. N. Abdul Rahman, and Z. B. Abal Abas, “Mosquito Larvae Detection using Deep Learning,” Int. J. Innov. Technol. Explor. Eng., vol. 8, no. 12, pp. 804–809, Oct. 2019.

R. Jain, P. Nagrath, G. Kataria, V. Sirish Kaushik, and D. Jude Hemanth, “Pneumonia detection in chest X-ray images using convolutional neural networks and transfer learning,” Measurement, vol. 165, p. 108046, Dec. 2020.

Y.-J. Cha, W. Choi, G. Suh, S. Mahmoudkhani, and O. Büyüköztürk, “Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types,” Comput. Civ. Infrastruct. Eng., vol. 33, no. 9, pp. 731–747, Sep. 2018.

M. Elleuch, R. Maalej, and M. Kherallah, “A New Design Based-SVM of the CNN Classifier Architecture with Dropout for Offline Arabic Handwritten Recognition,” Procedia Comput. Sci., vol. 80, pp. 1712–1723, 2016.

A. P. Wibawa, S. A. Kurniawan, and I. A. E. Zaeni, “Determining Journal Rank by Applying Particle Swarm Optimization-Naive Bayes Classifier,” J. Inf. Technol. Manag., vol. 13, no. 4, 2021.

T. B. Alakus and I. Turkoglu, “Comparison of deep learning approaches to predict COVID-19 infection,” Chaos, Solitons & Fractals, vol. 140, p. 110120, Nov. 2020.

DOI: http://dx.doi.org/10.17977/um018v5i12022p78-86