The Effect of Teachers' Training, Areas of Expertise, and Education Type on the Achievement of Multiple-choice HOTS Questions in Vocational High School

. This research aimed to find the impact of teachers' training, areas of expertise, and education type on the achievement of multiple-choice HOTS questions. The design of this research was experimental. The independent variable was training provision, the moderator variables were teachers' areas of expertise and education level, and the dependent variable was the achievement of multiple-choice HOTS questions. The subjects in this research were 28 teachers of Public Vocational High School 2 Turen consisting of teachers in productive and non-productive areas of expertise. The data collection used documentation and assignments. The data analysis technique used descriptive statistics with a paired t-test and factorial f-test. The significance level in this research was p = 0.05. Results showed that training provision significantly improved teacher's skills in making multiple-choice HOTS questions, improvement tendency of productive teachers after training compared to non-productive teachers, albeit insignificant, and improvement tendency shown by teachers from the non-teaching institutions after training compared to teachers from teaching institution, although also insignificant.


INTRODUCTION
signaled by information or concepts or knowledge application in situations familiar to students. Higherorder thinking involves solving assignments that have yet to be taught or known in a new context or case. Some researchers use Bloom's Taxonomy as the standard in assessing whether some questions are in LOTS or HOTS category. In Bloom's Taxonomy, LOTS questions cover knowledge and comprehension, while questions containing analysis, synthesis and evaluation are in the HOTS category. According to Thompson (2008), the application is often is found in LOTS and HOTS. Application can be included in the HOTS category when used in new situations.
Based on the above description, HOTS questions mean higher-order thinking questions consisting of application in new situations, analysis, synthesis, and evaluation. Or, according to the revised Bloom's Taxonomy, consists of applying in new condition, analysis, evaluation, and creating.
Training helps improve the skill in creating HOTS questions. Analysis showed that training provision significantly enhanced the workers' ability (Al Naqbi et al., 2018). Training also improved society's workability (Yudianto et al., 2020) substantially.
Variables that possibly influenced HOTS achievement in making multiple-choice questions were teachers' areas of expertise and education origin type. As commonly known, vocational high school teachers are divided into productive and non-productive and teachers who come from teaching and nonteaching institutions. Effective teachers teach practice and drawing subjects and rarely use multiplechoice questions, whereas non-productive teachers in theory subjects always use multiple-choice questions. Similarly, teachers from teaching institutions mainly receive materials on creating multiplechoice questions in HOTS; therefore, their skills in creating HOTS questions are better. Meanwhile, teachers from non-teaching institutions did not receive the material, reflecting poorly on their abilities. Thus, it could be said that teachers' areas of expertise and education origin type possibly influence the HOTS achievement in multiple-choice questions. This research aimed to find the influence of teachers' training, areas of expertise, and education type on the achievement of multiple-choice HOTS questions.

METHODS
This research was experimental design. The independent variable was training provision, the moderator variables were teachers' areas of expertise and education level, and the dependent variable was the achievement of multiple-choice HOTS questions. Hence, the control group was before training, and the experimental group was after training.
The subjects in this research were 28 teachers from Public Vocational High School 2 Turen, consisting of teachers in productive and non-productive areas of expertise. Data collection technique used documentation and assignments. Each subject made ten questions of HOTS before and after training on making multiple-choice questions HOTS. HOTS achievement was assessed using the sixlevel indicators of the revised edition of Bloom's Taxonomy. The research was conducted online and offline due to the pandemic.
The procedure was performed in eight stages. The first stage was creating and collecting data on multiple-choice HOTS questions that teachers made following their ability. The second stage was the evaluation of the questions. The third stage was sharing the material on how to create multiple-choice questions of HOTS. The fourth stage was an assignment on making multiple-choice HOTS questions. The fifth stage was the evaluation of the newest assignment. The sixth stage was comparing the initial questions with the after-training results. The seventh stage was comparing the HOTS achievement between productive and non-productive teachers and between teachers from teaching institutions and non-teaching institutions. The eighth stage found the interaction between training provision with areas of expertise and teachers' type of education institution and HOTS achievement.
The data analysis technique used descriptive statistics with paired t-test and factorial f-test. The descriptive analysis was utilized to find the central tendency of data before and after treatment. The mean difference analysis was used to see the difference and improvement from the before and after treatment, while factorial analysis of variance was utilized to obtain the interaction between areas of expertise and type of education with HOTS achievement. The requirements test is carried out before analyzing differences and interactions, resulting in a significance level of p = 0.05.

RESULT AND DISCUSSIONS Training Influence on HOTS Achievement
Before teachers' training, the mean value of HOTS was 55.57 and improved to 65.04 after training. Descriptively, there was a 9.47 points improvement or 17.04% from the initial value. The results of the data normality test showed that before training, data were not distributed normally with the value of Asymp. Sig. (2-tailed) 0.013 < 0.05. Meanwhile, after training, the data were distributed normally with the value of Asymp. Sig. (2-tailed) 0.200 > 0.05. This shows that there are data weaknesses in generalizing to the population.
The experimental analysis results showed a significant improvement in teachers' skills when creating multiple-choice questions of HOTS before and after training, obtaining the mean value of 55.57 before training and 65.04 after training. The factorial analysis of variance received F = 16.641 with significance 0.00 < 0.05 as written in Table 1. These results stated that training provision improved teachers' ability to create HOTS questions by 17.04%, similar to the previous research indicating that training provision improved teachers' ability to implement HOTS-based lessons by 4.23%. The improvement was significant (Sudan et al., 2020). Thus, the increase in the teacher's ability to make multiple choice HOTS questions after the training was significant. However, the ability has yet been optimized, with only scoring for 65.04%. This situation means that training needs to be done continuously to improve teachers' ability because HOTS questions making skill is essential in preparing exams.

Training and Areas of Expertise Influence on HOTS Achievement
The achievement of multiple-choice HOTS questions that teachers made is explained below. Descriptively, training improved teachers' ability in creating multiple-choice questions. However, there was a score difference between productive and non-productive teachers. Before training, the ability of productive teachers was lower than non-productive teachers. After training, they were better than nonproductive teachers, as seen in Table 2 with 13.46 points improvement or 25.03%, whereas the improved ability of non-productive teachers was 6.00 points or 10.50%. The t-test was performed to find the significant improvement after conducting the requirement test. Based on the normality test, both groups were not normally distributed. Based on the One-Sample Kolmogrov-Smirnov test, the Asymp. Sig. (2-tailed) < 0.05. These results indicated a weakness in the results of the analysis in generalization.
Mean difference analysis results showed no difference in teachers' ability in making HOTS questions between productive and non-productive. Results obtained non-homogenous t-test = 1.524 and significance of (2-tailed) = 0.143 > 0.05 as stated in Table 3. Hence, this research admitted that although productive teachers' ability in making HOTS questions was better than non-productive, the improvement was insignificant. Although, after training, productive teachers could create better results compared to prior training (Table 2), the interaction was insignificant. The interaction analysis between training provision and teachers' areas of expertise obtained the value of F = 2.446 and significance of 0.130 > 0.05, as shown in Table 4. Results implied that the interaction was limited and tended to be insignificant.  Figure 1 displays the interaction diagram of multiple-choice HOTS questions from before and after training those productive and non-productive teachers made. Before training, productive teachers made lower score questions than non-productive teachers, whereas, after training, they were better, although the interaction was insignificant.

Figure 1. Interaction Between Training Provision and Areas of Expertise
The insignificant interaction results could be caused by the sample of each cell that did not meet the total number; hence, even if interaction occurred descriptively, inferentially, it was insignificant. It can be explained that productive teachers with a lack of ability to create HOTS questions, marked by the low score (53.77), needed to improve themselves to become more confident. It became a motivation for those teachers to improve and better themselves. This situation followed previous analysis and research that stated a significant correlation between the need for self-actualization and work motivation, with r = 0.602 and p = 0.00 < 0.05. Higher need for self-actualization resulting in higher work motivation (Syauta & Yuniasanti, 2015). However, performance motivation and quality are also influenced by incentives. Different incentives might create different motivations and performances. Previous research showed that stimulus had a significant and positive effect on work motivation with p = 0.028 < 0.05 (Spanjaards, 2016;Graha, 2005). Likewise, according to Graha (2005), incentives positively and significantly impacted performance with p = 0.000 < 0.05. Therefore, based on previous researches, it can be concluded that they tended to have improvement on productive teachers after training, although insignificant.

Training and Education Origin Influence on HOTS Achievement
Before training, teachers from teaching institutions made higher HOTS questions than teachers from non-teaching institutions. After training, their skills were on par. The improvement, however, was higher on teachers from non-teaching institutions, as exhibited in Table 5. The improvement for teachers from teaching institutions was 7.10 points or 12.25%, while for teachers from non-teaching institutions was 14.44 points or 28.56%. These results showed that teachers from non-teaching teaching institutions adapted to HOTS questions making after training. Their initial lower skill in making HOTS questions became similar with the teachers from teaching institutions. The t-test was performed to find whether the improvement was significant or not after conducting the requirement test. The results of the normality test of data based on the type of education show that a group is normally distributed with significance (2-tailed) = 0.120 > 0.05, whereas another is not normally distributed with Asymp. Significance (2-tailed) = 0.001 < 0.05.
The mean difference analysis results showed no difference in teachers' ability in making HOTS questions between teaching institutions and non-teaching institutions. Results obtained nonhomogenous t-test = 1.430 and significance of (2-tailed) = 0.165 > 0.05 as stated in Table 6. Hence, this research admitted that although non-teaching institution-origin teachers' ability to make HOTS questions better than teachers from teaching institutions, the improvement was insignificant. As mentioned before training, teachers from non-teaching institutions had lower HOTS questions than teachers from teaching institutions. After training, they were on the same level as seen in Table 5. The interaction analysis between training provision and teachers' areas of expertise obtained the value of F = 2.046 and significance of 0.165 > 0.05, as shown in Table 7. Results implied that the interaction was insignificant.  Figure 2 displays the interaction diagram of multiple-choice HOTS questions from before and after training made by teachers from non-teaching institutions and teaching institutions. Before training, teachers from non-teaching institutions made lower score questions than teachers from teaching institutions, whereas, after training, they were almost the same.

Figure 2. Interaction Between Training Provision and Educational Type
The results can be explained as teachers who came from non-teaching institutions did not receive learning evaluation knowledge, and they felt the need to obtain that. The training material was needed to give good assessment during learning, and they could be more confident in making HOTS questions so that before training, they gained lower scores; after training, the score was balanced with teachers from teaching institutions. This condition followed previous research where self-esteem, selfactualization, and social needs affected motivation significantly. Social needs affected motivation by = 0.165 with p = 0.012, self-esteem needs affected motivation by = 0.321 with p = 0.000, and selfactualization needs affected motivation by = 0.243 with p = 0.001 (Taslimet al, 2018). Likewise, there was a significant positive relationship between self-actualization needs and work motivation, with r = 0.602 and p = 0.00 <0.05 (Syauta & Yuniasanti, 2015). However, the results showed that incentives influenced the quality of the performance. The provision of compensation or incentives had a positive and significant effect on employee performance (Oluigbo & Anyiam, 2014). There was a positive and significant effect between incentives on teacher performance (Maulana & Thohari, 2019& Pasaribu & Irsutami, 2015. Likewise, the study results found that partially material incentives affected motivation but had no effect on performance. This showed that if the incentives are not different, the quality of motivation and performance may also be similar. Therefore, based on previous researches, it can be concluded that they tended to have improvement on teachers from non-teaching institutions better than teacher institutions, although insignificant.

CONCLUSION
Based on the analysis and discussion, the conclusions were as below. First, the achievement in developing multiple-choice HOTS questions was 65.04, a 17.04% improvement from the initial score. Training provision on creating multiple-choice HOTS questions improved the level of the question significantly. Second, a tendency that the HOTS questions from productive teachers were better than non-productive teachers, although the improvement was insignificant. Descriptively, there was interaction in HOTS achievement with training provision in teachers' areas of expertise, although insignificant. Third, teachers from non-teaching institutions tend to improve better than teachers from teaching institutions, although minor. Likewise, there was no significant interaction between training provision and teachers' origin institution.