Development of Competency Test Instruments for PPG Students in the Automotive Field: Analysis of Validity, Reliability, and Discrimination Power

Syarif Suhartadi, Widiyanti Widiyanti

Abstract


This study aims to develop and validate a competency test instrument for students in the Teacher Professional Education Program (PPG) in the automotive field. The instrument was constructed through stages including the development of measurement specifications, question writing based on a blueprint, review, assembly, trial, analysis of trial results, as well as item selection and reassembly. The trial involved 73 PPG students at Universitas Negeri Malang. Item analysis covered difficulty level, discrimination power, validity, reliability, and distractor effectiveness. The results showed that out of 35 test items, 23 were deemed suitable for use after selection and revision. Thirty items had a medium level of difficulty, 5 were difficult, and none were easy. Regarding discrimination power, 22 items were categorized as ideal, 12 as low, and 1 as high. Item validity analysis revealed that only 9 items were valid, while the remaining 26 required revision or elimination. Distractor analysis showed 9 items with non-functioning distractors. The reliability of the reassembled instrument was 0.719, which is considered high according to Cronbach’s Alpha. These findings confirm that the developed instrument can be used to reliably measure the competencies of PPG automotive students. However, certain aspects, particularly the improvement of item discrimination, still require refinement. This research contributes to providing a valid, reliable evaluation tool aligned with the characteristics of PPG students and can serve as a reference for similar test development in other technical education fields.

Full Text:

PDF

References


Aayn, S. L., & Listiadi, A. (2022). The influence of school field introduction, teacher profession perception, and self-efficacy on readiness to become a teacher (Students of the Accounting Education Study Program at UNESA). Journal of Teacher and Learning Studies, 5(1), 132–140.

Achmad, I. A., Asmas, M. A., & Sukawati, Y. (2024). Evaluation of the Work Skills Education Program for Automotive Training in Improving Student Skills. Continuing Learning Society Journal, 2(1), 40–58.

Antonietti, C., Cattaneo, A., & Amenduni, F. (2022). Can teachers’ digital competence influence technology acceptance in vocational education? Computers in Human Behavior, 132, 107266.

Ardhani, Y. (2020). Quality of year-end exam items for Basic Automotive Technology, Grade X, Light Vehicle Engineering at SMK Muhammadiyah Gamping, 2018/2019. Journal of Vocational Automotive Education, 3(1), 85–94.

Botterbusch, K. F. (1987). Vocational Assessment and Evaluation Systems: A Comparison. ERIC.

Cattaneo, A. A. P., Antonietti, C., & Rauseo, M. (2022). How digitalised are vocational teachers? Assessing digital competence in vocational education and looking at its underlying factors. Computers & Education, 176, 104358.

Chikita, W. (2019). Analysis of item quality in the odd semester final exam for PKSM subject, Grade XI Motorcycle Engineering. Journal of Vocational Automotive Education, 2(1), 23–32.

Cronbach, L. J. (1949). Essentials of psychological testing.

Daniswara, A. A. A., & Nuryana, I. K. D. (2023). Data preprocessing patterns in student assessment of the Teacher Professional Education Program. Journal of Informatics and Computer Science (JINACS), 5(01), 97–100.

Daryono, R. W., Ramadhan, M. A., Kholifah, N., Isnantyo, F. D., & Nurtanto, M. (2023). An empirical study to evaluate the student competency of vocational education. International Journal of Evaluation and Research in Education (IJERE), 12(2), 1079–1086.

Diao, J., & Hu, K. (2022). Preparing TVET teachers for sustainable development in the information age: Development and application of the TVET teachers’ teaching competency scale. Sustainability, 14(18), 11361.

Hadi, S. (2012). Evaluation of the implementation of a competency-based curriculum in automotive course and training institutions in Yogyakarta. Journal of Vocational Education, 2(2), 267–283.

Hadi, S., Pd, M., Pidato, M. E., Jabatan, P., & Besar, G. (n.d.). Vocationalism–liberalism: Implications for governance and leadership of vocational education in Indonesia.

Hardiyanta, R. A. P., & Wagiran, W. (2023). Evaluation of competency tests in light vehicle engineering at vocational schools in Yogyakarta. Journal of Vocational Automotive Education, 5(2), 67–86.

Jatmoko, D., Basuki, B., Pambudi, A., Setiyono, K. A., & Nugroho, B. (2025). Analysis of basic automotive engineering items using cognitive diagnostic Anates Version 4.02. Journal of Educational Science Research, 4(1), 461–471.

Klotz, V. K., & Winther, E. (2017). Assessing Tomorrow’s Potential: a competence measuring approach in vocational education and training. In Competence Assessment in Education: Research, Models and Instruments (pp. 221–243). Springer.

Menchetti, B. M., & Rusch, F. R. (1988). Reliability and validity of the vocational assessment and curriculum guide. American Journal of Mental Retardation: AJMR, 93(3), 283–289.

Moktavia, M., Isnandar, I., & Ichwanto, M. A. (2024). Evaluation of block system learning in the Department of Modeling Design and Building Information at SMK Islam 1 Blitar. Proceedings of Life and Applied Sciences, 4(1).

Monnier, M., Tschöpe, T., Srbeny, C., & Dietzen, A. (2016). Occupation-specific social competences in vocational education and training (VET): the example of a technology-based assessment. Empirical Research in Vocational Education and Training, 8(1), 10.

Murwantini, S. (2023). Optimization of assessment for vocational high schools under the Independent Curriculum. Steam Engineering, 4(2), 105–113.

Pangestika, R. R. (2015). Teacher Professional Education (PPG): A strategy for developing teacher professionalism and improving the quality of Indonesian education.

Perdana, N. S. (2021). Analysis of student readiness in facing minimum competency assesment. MUKADIMAH: Jurnal Pendidikan, Sejarah, Dan Ilmu-Ilmu Sosial, 5(1), 15–20.

Priti, P., Sofyan, H., Budiman, A., & Sriyanto, J. (2023). Evaluation of industrial class programs in the Automotive Engineering major at SMK Negeri 2 Yogyakarta. Journal of Learning Innovation in Schools, 4(2), 602–610.

Samosir, D. M., Munawar, W., & Noor, R. A. M. (2024). Development of cognitive and psychomotor evaluation instruments for replacement of transaxle, motor, and generator. ATIKANOTO: Journal of Automotive Engineering Education, 1(1).

Sensus, M., Arifin, K., & Munir, A. (2022). Item validity in the Minimum Competency Assessment for Grade X High School Ecology. Wahana-Bio: Journal of Biology and Learning, 14(1), 1–10.

Subroto, J. G. (2019). Improving education quality: Teacher Professional Education (PPG) as pre-service professional teacher preparation in Indonesia. Scientific Article. http://download.garuda.kemdikbud.go.id/article.php

Venatius, A. S., Musta’amal, A. H., & Ekwok, O. B. (2023). Developing Framework for Teaching Technical Skills to Metalwork Undergraduates in Vocational-Based Education in Nigeria. Journal of Technical Education and Training, 15(2), 11–20. https://doi.org/10.30880/jtet.2023.15.02.002

Warju, W., Ariyanto, S. R., Soeryanto, S., & Trisna, R. A. (2020). Quality analysis of HOTS-type items in brake system competency at vocational high schools. Journal of Technology and Vocational Education, 17(1), 95–104.

Winther, E., & Klotz, V. K. (2013). Measurement of vocational competences: an analysis of the structure and reliability of current assessment practices in economic domains. Empirical Research in Vocational Education and Training, 5(1), 2.

Witarsa, B. N., Munawar, W., & Berman, E. T. (2017). Development and analysis of test items for Basic Automotive Technology in vocational schools. Journal of Mechanical Engineering Education, 4(2), 146–155.




DOI: http://dx.doi.org/10.17977/um010v8i22025p67-74

Refbacks

  • There are currently no refbacks.


 

 Letters in Information Technology Education (LITE)
 2654-5667
 Contact
 Published by Electrical Engineering, Universitas Negeri   Malang, Indonesia
 Jl. Semarang No. 5 Malang, East Java, Indonesia
 Homepage:  http://journal2.um.ac.id/index.php/lite/index
 E-mail: [email protected]

View My Stats