Optimizing Random Forest Algorithm to Classify Player's Memorisation via In-game Data

Akmal Vrisna Alzuhdi; Harits Ar Rosyid; Mohammad Yasser Chuttur; Shah Nazir

doi:10.17977/um018v6i12023p103-113

Optimizing Random Forest Algorithm to Classify Player's Memorisation via In-game Data

Akmal Vrisna Alzuhdi, Harits Ar Rosyid, Mohammad Yasser Chuttur, Shah Nazir

Abstract

Assessment of a player's knowledge in game education has been around for some time. Traditional evaluation in and around a gaming session may disrupt the players' immersion. This research uses an optimized Random Forest to construct a non-invasive prediction of a game education player's Memorization via in-game data. Firstly, we obtained the dataset from a 3-month survey to record in-game data of 50 players who play 4-15 game stages of the Chem Fight (a test case game). Next, we generated three variants of datasets via the preprocessing stages: resampling method (SMOTE), normalization (min-max), and a combination of resampling and normalization. Then, we trained and optimized three Random Forest (RF) classifiers to predict the player's Memorization. We chose RF because it can generalize well given the high-dimensional dataset. We used RF as the classifier, subject to optimization using its hyperparameter: n_estimators. We implemented a Grid Search Cross Validation (GSCV) method to identify the best value of n_estimators. We utilized the statistics of GSCV results to reduce the weight of n_estimators by observing the region of interest shown by the graphs of performances of the classifiers. Overall, the classifiers fitted using the BEST n_estimators (i.e., 89, 31, 89, and 196 trees) from GSCV performed well with around 80% accuracy. Moreover, we successfully identified the smaller number of n_estimators (OPTIMAL), at least halved the BEST n_estimators. All classifiers were retrained using the OPTIMAL n_estimators (37, 12, 37, and 41 trees). We found out that the performances of the classifiers were relatively steady at ~80%. This means that we successfully optimized the Random Forest in predicting a player's Memorization when playing the Chem Fight game. An automated technique presented in this paper can monitor student interactions and evaluate their abilities based on in-game data. As such, it can offer objective data about the skills used.

Full Text:

PDF

References

J. P. Gee, “What video games have to teach us about learning and literacy,” Comput. Entertain., vol. 1, no. 1, pp. 20–20, Oct. 2003.

J. J. C. U. Lee and J. C. U. Hammer, “Gamification in Education: What, How, Why Bother?,” Acad. Exch. Q., vol. 15, no. 2, pp. 1–5, 2011.

K. Squire and H. Jenkins, “Harnessing the power of games in education,” Insight, vol. 3, no. 1, pp. 5–33, 2003.

V. Shute et al., “Maximizing learning without sacrificing the fun: Stealth assessment, adaptivity and learning supports in educational games,” J. Comput. Assist. Learn., vol. 37, no. 1, pp. 127–141, Feb. 2021.

K. Mayfield et al., “Designing a Molecular Biology Serious Educational Game,” in Proceedings of the 2019 ACM Southeast Conference, Apr. 2019, pp. 210–213.

J. P. Gee, “Good video games and good learning,” Univ. Wisconsin–Madison. Recuper., 2009.

S. Çiftci, “Trends of Serious Games Research from 2007 to 2017: A Bibliometric Analysis.,” J. Educ. Train. Stud., vol. 6, no. 2, pp. 18–27, 2018.

O. Irmade, Suwarno, and N. Anisa, “Research Trends of Serious Games: Bibliometric Analysis,” J. Phys. Conf. Ser., vol. 1842, no. 1, p. 012036, Mar. 2021.

N. Kara, “Bibliometric and Content Analysis of Research Trends on the Use of Serious Games to Assist People with Disabilities,” J. Comput. Educ. Res., vol. 9, no. 17, pp. 278–299, Apr. 2021.

S. MEŞE and C. MEŞE, “Research Trends on Digital Games and Gamification in Nursing Education,” J. Comput. Educ. Res., vol. 10, no. 20, pp. 734–750, Dec. 2022.

S. P. Smith, K. Blackmore, and K. Nesbitt, “A Meta-Analysis of Data Collection in Serious Games Research,” in Serious Games Analytics, Cham: Springer International Publishing, 2015, pp. 31–55.

R. Aylett, M. Vala, P. Sequeira, and A. Paiva, “FearNot! – An Emergent Narrative Approach to Virtual Dramas for Anti-bullying Education,” in Virtual Storytelling. Using Virtual Reality Technologies for Storytelling, Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 202–205.

M. D. Kickmeier-Rust, C. Hockemeyer, D. Albert, and T. Augustin, “Micro Adaptive, Non-invasive Knowledge Assessment in Educational Games,” in 2008 Second IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning, 2008, pp. 135–137.

M. D. Kickmeier-Rust, An alien’s guide to multi-adaptive educational computer games. Informing Science, 2012.

M. D. Kickmeier-Rust, C. M. Steiner, and D. Albert, “Non-invasive Assessment and Adaptive Interventions in Learning Games,” in 2009 International Conference on Intelligent Networking and Collaborative Systems, Nov. 2009, pp. 301–305.

B. Magerko, C. Heeter, J. Fitzgerald, and B. Medler, “Intelligent adaptation of digital game-based learning,” in Proceedings of the 2008 Conference on Future Play: Research, Play, Share, Nov. 2008, pp. 200–203.

F. Bellotti, R. Berta, A. De Gloria, and L. Primavera, “Adaptive Experience Engine for Serious Games,” IEEE Trans. Comput. Intell. AI Games, vol. 1, no. 4, pp. 264–280, Dec. 2009.

A. Plotnikov et al., “Measuring enjoyment in games through electroencephalogram (eeg) signal analysis,” in Proceedings of the 6th European Conference on Games-Based Learning (ECGBL 2012), 2012, pp. 393–400.

os, K. Lee, P. Moreno-Ger, and R. Berta, “Assessment in and of Serious Games: An Overview,” Adv. Human-Computer Interact., vol. 2013, pp. 1–11, 2013.

D. Ismailović, J. Haladjian, B. Köhler, D. Pagano, and B. Brügge, “Adaptive serious game development,” in 2012 Second International Workshop on Games and Software Engineering: Realizing User Engagement with Game Engineering Techniques (GAS), 2012, pp. 23–26.

B. Magerko, B. Stensrud, and L. Holt, “Bringing the schoolhouse inside the box-a tool for engaging, individualized training,” 2006.

J. Rowe and J. Lester, “Modeling User Knowledge with Dynamic Bayesian Networks in Interactive Narrative Environments,” Proc. AAAI Conf. Artif. Intell. Interact. Digit. Entertain., vol. 6, no. 1, pp. 57–62, Oct. 2010.

T. Hainey et al., “Serious games as innovative formative assessment tools for programming in Higher Education,” 2022.

P. F. Lazarsfeld, “The language of social research: A reader in the methodology of social research,” 1964.

F.-L. Fu, R.-C. Su, and S.-C. Yu, “EGameFlow: A scale to measure learners’ enjoyment of e-learning games,” Comput. Educ., vol. 52, no. 1, pp. 101–112, Jan. 2009.

R. Lopes and R. Bidarra, “Adaptivity Challenges in Games and Simulations: A Survey,” IEEE Trans. Comput. Intell. AI Games, vol. 3, no. 2, pp. 85–99, Jun. 2011.

C. Alonso-Fernández, M. Freire, I. Martínez-Ortiz, and B. Fernández-Manjón, “Improving evidence-based assessment of players using serious games,” Telemat. Informatics, vol. 60, p. 101583, Jul. 2021.

Devottam Gaurav, Yash Kaushik, Santhoshi Supraja, Manav Yadav, M P Gupta, and Manmohan Chaturvedi, “Empirical Study of Adaptive Serious Games in Enhancing Learning Outcome,” Int. J. Serious Games, vol. 9, no. 2, pp. 27–42, May 2022.

A. S. Osman, “Data Mining Techniques: Review. 2 (1), 1–4.” 2019.

J. Han, M. Kamber, and J. Pei, “Data mining, southeast asia edition: Concepts and techniques. 2006.” Morgan kaufmann.

J. A. Caballero-Hernández, M. Palomo-Duarte, J. M. Dodero, and D. Gaševic, “Supporting Skill Assessment in Learning Experiences Based on Serious Games Through Process Mining Techniques,” Int. J. Interact. Multimed. Artif. Intell., vol. In Press, no. In Press, p. 1, 2023.

S. Abbasi and H. Kazi, “Stealth assessment in serious games to improve OO learning outcomes,” in 2019 International Conference on Advances in the Emerging Computing Technologies (AECT), Feb. 2020, pp. 1–5.

M. Manske and C. Conati, “Modelling Learning in an Educational Game,” in AIED, 2005, vol. 2005, pp. 411–418.

Z. Feng, Y. Shi, D. Zhou, and L. Mo, “Research on Human Activity Recognition Based on Random Forest Classifier,” in 2023 IEEE International Conference on Control, Electronics and Computer Technology (ICCECT), Apr. 2023, pp. 1507–1513.

H. A. Rosyid, M. Palmerlee, and K. Chen, “Deploying learning materials to game content for serious education game development: A case study,” Entertain. Comput., vol. 26, pp. 1–9, May 2018.

L. Bao, “Dynamic models of learning and education measurement,” arXiv Prepr. arXiv0710.1375, 2007.

A. Statnikov, L. Wang, and C. F. Aliferis, “A comprehensive comparison of random forests and support vector machines for microarray-based cancer classification,” BMC Bioinformatics, vol. 9, no. 1, p. 319, Dec. 2008.

T. G. Dietterich, “Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms,” Neural Comput., vol. 10, no. 7, pp. 1895–1923, Oct. 1998.

DOI: http://dx.doi.org/10.17977/um018v6i12023p103-113