A Comparison of Machine Learning Models to Prioritise Emails  using Emotion Analysis for Customer Service Excellence

Mohammad Yasser Chuttur; Yashinee Parianen

doi:10.17977/um018v5i12022p41-52

A Comparison of Machine Learning Models to Prioritise Emails using Emotion Analysis for Customer Service Excellence

Mohammad Yasser Chuttur, Yashinee Parianen

Abstract

There has been little research on machine learning for email prioritization for customer service excellence. To fill this gap, we propose and assess the efficacy of various machine learning techniques for classifying emails into three degrees of priority: high, low, and neutral, based on the emotions inherent in the email content. It is predicted that after emails are classified into those three categories, recipients will be able to respond to emails more efficiently and provide better customer service. We use the NRC Emotion Lexicon to construct a labeled email dataset of 517,401 messages for our proposal. Following that, we train and test four prominent machine learning models, MNB, SVM, LogR, and RF, and an Ensemble of MNB, LSVC, and RF classifiers, on the labeled dataset. Our main findings suggest that machine learning may be used to classify emails based on their emotional content. However, some models outperform others. During the testing phase, we also discovered that the LogR and LSVC models performed the best, with an accuracy of 72%, while the MNB classifier performed the poorest. Furthermore, classification performance differed depending on whether the dataset was balanced or imbalanced. We conclude that machine learning models that employ emotions for email classification are a promising avenue that should be explored further.

Full Text:

PDF

References

B. Graf and C. H. Antoni, “The relationship between information characteristics and information overload at the workplace-a meta-analysis,” European Journal of Work and Organizational Psychology, vol. 30, no. 1, pp. 143–158, 2021.

B. Mannion, “Information Overload,” Risk Management, vol. 69, no. 4, pp. 26–29, 2022.

R. Kong, H. Zhu, and J. A. Konstan, “Learning to ignore: A case study of organization-wide bulk email effectiveness,” Proceedings of the ACM on Human-Computer Interaction, vol. 5, no. CSCW1, pp. 1–23, 2021.

T. Ravichandran and C. Deng, “Effects of Managerial Response to Negative Reviews on Future Review Valence and Complaints,” Information Systems Research, 2022.

E. Russell, S. A. Woods, and A. P. Banks, “Tired of email? Examining the role of extraversion in building energy resources after dealing with work-email,” European journal of work and organizational psychology, vol. 31, no. 3, pp. 440–452, 2022.

Z. Halim, M. Waqar, and M. Tahir, “A machine learning-based investigation utilizing the in-text features for the identification of dominant emotion in an email,” Knowledge-Based Systems, vol. 208, p. 106443, Nov. 2020.

Z. Shao, R. Chandramouli, K. P. Subbalakshmi, and C. T. Boyadjiev, “An analytical system for user emotion extraction, mental state modeling, and rating,” Expert Systems with Applications, vol. 124, pp. 82–96, Jun. 2019.

X. Li and R. Lin, “Speech Emotion Recognition for Power Customer Service,” in 2021 7th International Conference on Computer and Communications (ICCC), 2021, pp. 514–518.

S. Angel Deborah, T. T. Mirnalinee, and S. M. Rajendram, “Emotion analysis on text using multiple kernel gaussian...,” Neural Processing Letters, vol. 53, no. 2, pp. 1187–1203, 2021.

M. Haberzettl and B. Markscheffel, “A Literature Analysis for the Identification of Machine Learning and Feature Extraction Methods for Sentiment Analysis,” in 2018 Thirteenth International Conference on Digital Information Management (ICDIM), Sep. 2018, pp. 6–11.

Y. Chuttur and L. Pokhun, “An Evaluation of Deep Learning Networks to Extract Emotions from Yelp Reviews,” in Progress in Advanced Computing and Intelligent Engineering, Springer, 2021, pp. 55–67.

L. Pokhun and M. Y. Chuttur, “Emotions in texts,” Bulletin of Social Informatics Theory and Application, vol. 4, no. 2, pp. 59–69, 2020.

V. Ahire and S. Borse, “Emotion detection from social media using machine learning techniques: a survey,” in Applied Information Processing Systems, Springer, 2022, pp. 83–92.

R. Plutchik, “The nature of emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice,” American scientist, vol. 89, no. 4, pp. 344–350, 2001.

A. A. Alurkar et al., “A proposed data science approach for email spam classification using machine learning techniques,” in 2017 Internet of Things Business Models, Users, and Networks, Nov. 2017, pp. 1–5.

S. R. Gomes et al., “A comparative approach to email classification using Naive Bayes classifier and hidden Markov model,” in 2017 4th International Conference on Advances in Electrical Engineering (ICAEE), Sep. 2017, pp. 482–487.

E. G. Dada, J. S. Bassi, H. Chiroma, S. M. Abdulhamid, A. O. Adetunmbi, and O. E. Ajibuwa, “Machine learning for email spam filtering: review, approaches and open research problems,” Heliyon, vol. 5, no. 6, p. e01802, Jun. 2019.

F. Jáñez-Martino, E. Fidalgo, S. González-Martínez, and J. Velasco-Mata, “Classification of Spam Emails through Hierarchical Clustering and Supervised Learning,” arXiv:2005.08773 [cs], May 2020, Accessed: Dec. 12, 2020.

S. Liu and I. Lee, “Email Sentiment Analysis Through k-Means Labeling and Support Vector Machine Classification,” Cybernetics and Systems, vol. 49, no. 3, pp. 181–199, Apr. 2018.

R. S. H. Ali and N. E. Gayar, “Sentiment Analysis using Unlabeled Email data,” in 2019 International Conference on Computational Intelligence and Knowledge Economy (ICCIKE), Dec. 2019, pp. 328–333.

N. Saidani, K. Adi, and M. S. Allili, “A semantic-based classification approach for an enhanced spam detection,” Computers & Security, vol. 94, p. 101716, Jul. 2020.

N. Ahmed, R. Amin, H. Aldabbas, D. Koundal, B. Alouffi, and T. Shah, “Machine learning techniques for spam detection in email and IoT platforms: analysis and research challenges,” Security and Communication Networks, vol. 2022, 2022.

R. Mansoor, N. D. Jayasinghe, and M. M. A. Muslam, “A comprehensive review on email spam classification using machine learning algorithms,” in 2021 International Conference on Information Networking (ICOIN), 2021, pp. 327–332.

I. Amin and M. K. Dubey, “Hybrid ensemble and soft computing approaches for review spam detection on different spam datasets,” Materials Today: Proceedings, 2022.

P. Garg and N. Girdhar, “A Systematic Review on Spam Filtering Techniques based on Natural Language Processing Framework,” in 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence), 2021, pp. 30–35.

B. Wang, “Personalized Broadcast Message Prioritization,” Thesis, Applied Sciences: School of Computing Science, 2018. Accessed: Jan. 10, 2021.

S. Choudhari, N. Choudhary, S. Kaware, and A. Shaikh, “Email Prioritization Using Machine Learning,” SSRN Journal, 2020.

E. M. Bahgat, S. Rady, and W. Gad, “An Email Filtering Approach Using Classification Techniques,” in The 1st International Conference on Advanced Intelligent System and Informatics (AISI2015), November 28-30, 2015, Beni Suef, Egypt, Cham, 2016, pp. 321–331.

N. Chhaya, K. Chawla, T. Goyal, P. Chanda, and J. Singh, “Frustrated, Polite, or Formal: Quantifying Feelings and Tone in Email,” in Proceedings of the Second Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media, New Orleans, Louisiana, USA, Jun. 2018, pp. 76–86.

E. M. Bahgat, S. Rady, W. Gad, and I. F. Moawad, “Efficient email classification approach based on semantic methods,” Ain Shams Engineering Journal, vol. 9, no. 4, pp. 3259–3269, Dec. 2018.

M. A. Naser and A. H. Mohammed, “Emails classification by data mining techniques,” Journal of Babylon University: Pure and Applied Sciences, Vol. 22, No 2, 2014.

X. Fang and J. Zhan, “Sentiment analysis using product review data,” J Big Data, vol. 2, Dec. 2015.

Xiao-Lin Wang and Cloete, “Learning to classify email: a survey,” in 2005 International Conference on Machine Learning and Cybernetics, Aug. 2005, vol. 9, pp. 5716-5719 Vol. 9.

B. Ray, A. Garain, and R. Sarkar, “An ensemble-based hotel recommender system using sentiment analysis and aspect categorization of hotel reviews,” Applied Soft Computing, vol. 98, p. 106935, Jan. 2021.

P. Barnaghi, P. Ghaffari, and J. G. Breslin, “Opinion Mining and Sentiment Polarity on Twitter and Correlation between Events and Sentiment,” in 2016 IEEE Second International Conference on Big Data Computing Service and Applications (BigDataService), Mar. 2016, pp. 52–57.

V. L. Miguéis, A. Freitas, P. J. V. Garcia, and A. Silva, “Early segmentation of students according to their academic performance: A predictive modelling approach,” Decision Support Systems, vol. 115, pp. 36–51, Nov. 2018.

Q. Umer, H. Liu, and Y. Sultan, “Emotion Based Automated Priority Prediction for Bug Reports,” IEEE Access, vol. 6, pp. 35743–35752, 2018.

T. Bokaba, W. Doorsamy, and B. S. Paul, “Comparative Study of Machine Learning Classifiers for Modelling Road Traffic Accidents,” Applied Sciences, vol. 12, no. 2, Art. no. 2, Jan. 2022.

K. Y. Win, N. Maneerat, S. Choomchuay, S. Sreng, and K. Hamamoto, “Suitable Supervised Machine Learning Techniques For Malignant Mesothelioma Diagnosis,” in 2018 11th Biomedical Engineering International Conference (BMEiCON), Nov. 2018, pp. 1–5.

C. K. Hiramath and G. C. Deshpande, “Fake News Detection Using Deep Learning Techniques,” in 2019 1st International Conference on Advances in Information Technology (ICAIT), Jul. 2019, pp. 411–415.

D. K. Renuka, T. Hamsapriya, M. R. Chakkaravarthi, and P. L. Surya, “Spam Classification Based on Supervised Learning Using Machine Learning Techniques,” in 2011 International Conference on Process Automation, Control and Computing, Jul. 2011, pp. 1–7.

M. B. Abbas and M. Khan, “Sentiment Analysis for Automated Email Response System,” in 2019 International Conference on Communication Technologies (ComTech), Mar. 2019, pp. 65–70.

DOI: http://dx.doi.org/10.17977/um018v5i12022p41-52