Rasch Model: Analysing the Items Quality of Mathematics Higher-Order Thinking Skill Instrument

Muhamad Syahidul Qirom; Elah Nurlaelah

doi:10.17977/um047v30i12023p8-16

Rasch Model: Analysing the Items Quality of Mathematics Higher-Order Thinking Skill Instrument

Muhamad Syahidul Qirom, Elah Nurlaelah

Abstract

This research aims to analyse a mathematics HOTS instrument using the Rasch Model to provide a better measurement instrument for mathematics. The result showed, based on Rasch Model analysis, 1) from 20 items, there are only ten items that can be used to measure HOTS of students in mathematics with the difficulty of 3 items being hard and seven items being easy; 2) the reliabilities of the instrument are moderate; 3) the instrument only can divide students into two groups of level ability; and 4) there is no significant differential item functioning (DIF) bias detected on the instrument.

Keywords

Rasch Model, HOTS, Mathematics, Item Quality, Assessment and Evaluation

Full Text:

PDF

References

Anderson, L. W., Krathwhol, D. R., Airasian, P., Cruikshank, K., Mayer, R., Pintrich, P., … Wittrock, M. (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objective. New York: Addison Wesley Longman.

Bichi, A. A., Talib, R., Embong, R., & Salleh, S. salleh. (2019). Comparative Analysis of Classical Test Theory and Item Response Theory using Chemistry Test Data. International Journal of Engineering and Advanced Technology, 8(5C), 1260–1266. https://doi.org/10.35940/ijeat.E1179.0585C1

Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (Third edition). New York: Routledge.

Boone, W. J. (2016). Rasch Analysis for Instrument Development: Why, When, and How? CBE—Life Sciences Education, 15(4), rm4. https://doi.org/10.1187/cbe.16-04-0148

Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch Analysis in the Human Sciences. Dordrecht: Springer Netherlands. https://doi.org/10.1007/978-94-007-6857-4

Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. Alexandria, Va: ASCD.

Butterworth, J., & Thwaites, G. (2013). Thinking Skills: Critical Thinking and Problem Solving (Secon Edition). Cambridge: Cambridge University Press.

Chiruguru, S. (2020). The Essential Skills of 21st Century Classroom (4Cs). https://doi.org/10.13140/RG.2.2.36190.59201

Gay, L. R., Mills, G. E., & Airasian, P. W. (2012). Educational research: Competencies for analysis and applications (10th ed). Boston: Pearson.

Hambleton, R. K., & Jones, R. W. (2005). An NCME Instructional Module on: Comparison of Classical Test Theory and Item Response Theory and Their Applications to Test Development. Educational Measurement: Issues and Practice, 12(3), 38–47. https://doi.org/10.1111/j.1745-3992.1993.tb00543.x

Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, Calif: Sage Publications.

Haw, L. H., Sharif, S. B., & K. Han, C. G. (2022). Analyzing the science achievement test: Perspective of classical test theory and Rasch analysis. International Journal of Evaluation and Research in Education (IJERE), 11(4), 1714. https://doi.org/10.11591/ijere.v11i4.22304

Hinton, P. R., McMurray, I., & Brownlow, C. (2014). SPSS explained (Second edition). London: Routledge.

How, R. P. T. K., Zulnaidi, H., & Rahim, S. S. B. A. (2023). Development of Higher-Order Thinking Skills test instrument on Quadratic Equation (HOTS-QE) for secondary school students. Pegem Journal of Education and Instruction, 13(1). https://doi.org/10.47750/pegegog.13.01.41

Jabrayilov, R., Emons, W. H. M., & Sijtsma, K. (2016). Comparison of Classical Test Theory and Item Response Theory in Individual Change Assessment. Applied Psychological Measurement, 40(8), 559–572. https://doi.org/10.1177/0146621616664046

Karlimah, K. (2022). How does Rasch modeling reveal difficulty and suitability level the fraction test question? Jurnal Elemen, 8(1), 66–76. https://doi.org/10.29408/jel.v8i1.4170

Karlin, O., & Karlin, S. (2018). Making Better Tests with the Rasch Measurement Model. InSight: A Journal of Scholarly Teaching, 13, 76–100. https://doi.org/10.46504/14201805ka

Linacre, J. M. (2022). A User’s Guide to WINSTEP-MINISTEP: Rasch-model computer program. Retrieved from winsteps.com

Mertler, C. A. (2015). Quantitative Research Methods. In Introduction to Educational Research (First Edition). California: SAGE Publications.

Michalos, A. C. (Ed.). (2014a). Encyclopedia of Quality of Life and Well-Being Research. Dordrecht: Springer Netherlands. https://doi.org/10.1007/978-94-007-0753-5

Michalos, A. C. (Ed.). (2014b). Rasch Analysis. In Encyclopedia of Quality of Life and Well-Being Research (pp. 5393–5395). Dordrecht: Springer Netherlands. https://doi.org/10.1007/978-94-007-0753-5_2394

Nassaji, H. (2015). Qualitative and descriptive research: Data type versus data analysis. Language Teaching Research, 19(2), 129–132. https://doi.org/10.1177/1362168815572747

Petra, T. Z. H. T., & Aziz, M. J. A. (2020). Investigating reliability and validity of student performance assessment in Higher Education using Rasch Model. Journal of Physics: Conference Series, 1529(4), 042088. https://doi.org/10.1088/1742-6596/1529/4/042088

Qirom, M. S., Sridana, N., & Prayitno, S. (2020). Pengembangan Soal Matematika Berbasis Higher Order Thinking Skills Pada Lingkup Materi Ujian Nasional Untuk Tingkatan Sekolah Menengah Pertama. Jurnal Pijar Mipa, 15(5), 466–472. https://doi.org/10.29303/jpm.v15i5.2028

Quaigrain, K., & Arhin, A. K. (2017). Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4(1), 1301013. https://doi.org/10.1080/2331186X.2017.1301013

Scott, N. W., Fayers, P. M., Aaronson, N. K., Bottomley, A., de Graeff, A., Groenvold, M., … Sprangers, M. A. (2010). Differential item functioning (DIF) analyses of health-related quality of life instruments using logistic regression. Health and Quality of Life Outcomes, 8(1), 81. https://doi.org/10.1186/1477-7525-8-81

Sürücü, L., & Maslakçi, A. (2020). Validity and reliability in quantitative research. Business & Management Studies: An International Journal, 8(3), 2694–2726. https://doi.org/10.15295/bmij.v8i3.1540

Tavakol, M., & Dennick, R. (2013). Psychometric evaluation of a knowledge based examination using Rasch analysis: An illustrative guide: AMEE Guide No. 72. Medical Teacher, 35(1), e838–e848. https://doi.org/10.3109/0142159X.2012.737488

Yudha, R. P. (2023). Higher Order Thinking Skills (HOTS) Test Instrument: Validity and Reliability Analysis With The Rasch Model. EduMa: Mathematics Education Learning And Teaching, 12(1), 21–38. http://dx.doi.org/10.24235/eduma.v12i1.9468

Zamora-Araya, J. A., Smith-Castro, V., Montero-Rojas, E., & Moreira-Mora, T. E. (2018). Advantages of the Rasch Model for Analysis and Interpretation of Attitudes: The Case of the Benevolent Sexism Subscale. Revista Evaluar, 18(3). https://doi.org/10.35670/1667-4545.v18.n3.22201

DOI: http://dx.doi.org/10.17977/um047v30i12023p8-16