Online Learning Evaluation of Mathematics Using the CIPP Model

The purpose of this study was to analyze and evaluate the implementation of online learning, especially in Mathematics, by applying one of the evaluation models, namely the CIPP (Context, Input, Process, Product). The approach of this research is qualitative with the type of evaluation research. The research data sources consist of primary data, namely in the form of notes from interviews with several informants considered credible in providing data and those obtained from direct observation of the teaching and learning activities. The instruments employed in this study were interview guides, observation sheets, and documentation. Based on the data exposure, it was concluded that the implementation of the learning process was quite effective, including the teacher's workload, the number of study groups, also a complete and well-organized class management. The findings reveal that the less optimal learning time management to achieve the learning objectives needs evaluation. In addition, the assessment stage, which is part of the learning process, was not right on target.


INTRODUCTION
Since December 2019, Indonesia and even the world have been shocked by the spread of the coronavirus. This variant of the coronavirus is new, which is a non-natural disaster and causes massive transmission. Even, in April 2021, the coronavirus has spread in several regions in Indonesia. The coronavirus has a negative impact, namely, on the instability of a country seen from various aspects (Prisuna, 2021a). Recent research shows that the coronavirus pandemic is sending shockwaves throughout the education system on an unprecedented global scale (Johnson et al., 2020). One of the disastrous impacts of the coronavirus is in education, experienced by most students in Indonesia.
Various efforts to prevent the transmission of this virus have been made by the government of Indonesia. One of them is PPKM, which stands for Community Activities Restrictions Enforcement (CARE), intending to provide rules related to all forms of community activities, including teaching and learning activities. Pontianak City itself is one of those implementing CARE level 3. This policy starts on August 10, 2021, and is stated in the Instruction of the Minister of Home Affairs No. 32 of 2021. In the instruction, the Ministry of Home Affairs conveyed several provisions during the implementation of CARE level 3, one of which was related to the mechanism for implementing limited face-to-face learning; such as distance learning.
Online learning has increased with the technology development , alongside the Coronavirus Disease  pandemic. Online learning is a learning process in which education takes place totally using the internet (Hijazi & Alnatour, 2021). It is in line with the opinion of Jayul & Irwanto (2020) that online learning refers to a teaching and learning activity that utilizes the internet as an intermediary between students and teachers. As to Kuntarto (2017), teaching and learning interaction activities between lecturers and students with the help of the internet are considered online learning. Online learning requires media as a means of learning in schools; undoubtedly, in terms of various applications to facilitate the delivery of learning materials. Among the apps are Zoom, Google Classroom, Jitsi Meet, Google Meet, WhatsApp, etc. (Haqien & Rahman, 2020). It is a challenge for educators to see the pandemic conditions that have more or less affected the learning process. Answering the phenomena that occur, the teacher is expected to be a problem solver (Prisuna, 2021b). It is to create conducive teaching and learning activities so that learning objectives can be achieved, especially in the learning process of Mathematics subjects contained in Permendikbud No.22 of 2016 regarding the objectives of learning mathematics. The four objectives are understanding mathematical concepts as a whole, examining patterns of characteristics in mathematics, solving various mathematical problems, and communicating opinions or arguments. Quality mathematics teaching is the key to achieving mathematics learning objectives, especially during the current COVID-19 pandemic, where almost all learning activities are online.
The goals of online learning include providing quality online learning services that can accommodate a larger number of students with a broad reach (Bilfaqih & Qomarudin, 2015). Seeing the current pandemic phenomenon, education policymakers have to think hard and smart about an effective approach to learning mathematics to achieve the goals of learning it.
The results of the researcher's interview on September 7, 2021, with the Mathematics teacher at SMPIT Al-Mumtaz Pontianak regarding the implementation of online learning, are elaborated as follows. The implementation of online learning was carried out fully online between students and teachers from 07.30-11.00 WIB. The media that must be used in the online learning process according to the instructions from the curriculum representative was Google Classroom. All teachers and students were facilitated by a curriculum representative regarding the creation of emails to activate their Google Classroom accounts. The Mathematics teacher at SMPIT Al-Mumtaz added, "as the additional media, especially in Mathematics classes, are Youtube videos, pointers in PowerPoint, etc.". According to him, so far, the online learning conducted at SMPIT Al-Mumtaz Pontianak had been quite neat and effective. However, it is necessary to measure the effectiveness of the level of success so that it does not become a subjective assessment. To measure the extent to which the learning objectives are achieved, an evaluation must be carried out.
According to Uno & Koni (2013), evaluation means a series of activities giving meaning or determining the quality of a measurement result by comparing several measurement results with predetermined criteria. Evaluation is the center of all forms of improvement, especially in the quality of education (Aziz et al., 2018). Meanwhile, according to Wandt & Brown in Ratnawulan & Rusdiana (2014), evaluation produces something resulted from an assessment process. Based on these definitions, evaluation refers to something as a process/an action to produce value. An education system designed in such a way and its implementation will definitely have advantages and disadvantages. The purpose of the evaluation is to analyze more deeply the quality of the system that has been built. In line with Law No. 20 of 2003 concerning National Education System, which is stated in article 57 paragraph (1) and (2), which essentially evaluates the purpose of quality control in an institution at all levels of education, evaluation in education aims to maintain the quality of an educational program itself. According to Lync in Erdogan & Mede (2021), program evaluation in education is a process of collecting, analyzing, and interpreting something related to teaching and learning to determine the value of the program. Various designs or evaluation models have been used, and the CIPP model is an evaluation model that is generally used in the education sector. Stufflebeam was the first to develop the CIPP model in 1966. The CIPP model is one of the most widely used evaluation approaches and has been successful in its application, starting from evaluating the curriculum at the kindergarten level (Al-Shanawani, 2019), at the elementary school level (Luma et al., 2020), at the Junior High School level (Raibowo & Nopiyanto, 2020), at the high school level (Batdı, 2017) up to the college level (Uğur et al., 2016). This model embodies and builds on the definition of evaluation in general. The CIPP model provides an overview of the assessment of the aspects of context, input, process, and product (Stufflebeam & Zhang, 2017). Evaluation is an activity to describe, receive, and present data to become an alternative to a decision (Wirawan, 2016).
The CIPP model is an evaluation model for curriculum evaluation provided by Stufflebeam in 1983 that includes four elements: (1) Context, covering the objectives, history and background of the institution; (2) Input, covering human resources in this case are the students and teachers as well as matters relating to the ongoing operation of an institution; (3) Process, covering all learning processes; and (4) Product, covering the quality of teaching and learning as well as using values and benefits for the surrounding environment (Stufflebeam, 2003). This model is considered effective to assess the quality of education. This view is shared by Agustina & Mukhtaruddin (2019) that The CIPP model promoted by Stufflebeam is one of many models suitable for evaluation and assessment activities.
Starting from the discourse that learning in the future will continue to develop by utilizing technology, it is for this reason that this research was carried out, namely to analyze and evaluate the implementation of online learning, especially in mathematics studies at SMPIT Al-Mumtaz Pontianak. It is hoped that the evaluation results from the research can provide appropriate recommendations in improving the quality of education and learning, especially at the Al-Mumtaz SMPIT Pontianak School and other schools in general.

METHOD
This study uses a qualitative approach with the type of evaluation research. This research was carried out in the odd semester of the 2020/2021 academic year at SMPIT Al-Mumtaz Pontianak. The research data consist of the primary and secondary data. The primary data are notes from interviews with the principals, educators, students, and students' parents, also from direct observations of the online learning process at SMPIT Al-Mumtaz Pontianak, especially the Mathematics classes. Then, the secondary data are in the form of all documents that support the primary data.
The data collection instruments were interview guidelines with semi-structured and unstructured questions posed to the principals, curriculum representatives, educators (Math teachers), students, and the students' parents/guardians to obtain information on online learning at the school. The online observation protocol, referred to research conducted by Surahman & Pratama (2021), includes several aspects of observation, namely (1) the teaching process, (2) student learning activities, (3) assessment of the learning process, and (4) learning platform. In this study, all activities related to online learning at SMPIT Al-Mumtaz Pontianak were observed, and the documents containing the profile of the school, the basis for the implementation of the online learning, the online learning planning, the reports on the online learning implementation, plans for activities that take place for students during the online learning, data on educators and students of SMPIT Al-Mumtaz Pontianak, and the results of learning outcomes obtained by students during the online learning were collected. The CIPP Model used to evaluate the online learning is shown in Figure 1. These interrelationships are made functional by basing evaluations on core values, as represented by circles in the schema (Stufflebeam, 2003).
The data analysis technique used the Miles and Huberman model, of which stages were: data collection, data reduction, data presentation, and conclusion drawing. In this research, some tests as the techniques of checking the validity of the data were applied, which include: a credibility test, transferability test, dependability test, and confirmability test.

Evaluation of the aspect of context
Evaluation of the context aspect begins with the profile of SMPIT Al-Mumtaz which is located on Jl. Kemakmuran, Sui Jawi, Pontianak City District, Pontianak City, to be precise in West Kalimantan with post code 78113. The vision of the school is to realize SMPIT Al-Mumtaz as the front line contributor to civilization in the West Kalimantan region. The missions are to a) integrate methodology, curriculum, and sustainable Islamic education programs; b) create students with achievements, pious, and istiqamah characters who are ready to compete in further education; c) provide education that is oriented to the development of student's competencies; d) foster an academic culture; and e) improve the relationship in the surrounding environment, with parents, society, etc.

Evaluation of the input aspect
Evaluation on the input aspect in the online learning program at SMPIT Al-Mumtaz includes: students, number of groups, teachers, curriculum, teaching materials, and facilities & infrastructure as described in the detail explanation below. Some of the input aspects can be seen from the description of the primary data in Table 1. The paramount teaching materials at SMPIT Al-Mumtaz are textbooks prepared by the government, which were intended for students to learn in face-to-face learning conditions. A total of 214 types of reading books are available in the library. There are 17 books related to mathematics. In addition to printed books, SMPIT Al-Mumtaz also uses supporting learning media, including: video player, audio player, and powerpoint presentations. The supporting teaching materials in mathematics are in the form of simple game tools that require students to use problem solving skills. e. Teacher There are 29 teachers at SMPIT Al-Mumtaz, and five of them are with math qualifications. Five of them have bachelor degree in mathematic education. On average, teachers at Al-Mumtaz have more than one year of teaching experience. Various training activities related to quality improvement are held almost every year. f. Facilities and infrastructure The evaluation results show that the study room used by SMPIT Al-Mumtaz is a school building. Some of the data from the facilities and infrastructure at SMPIT Al-Mumtaz are presented in the Table 2. In detail, the facilities and infrastructure used the source of electricity from State Electricity Company with a power capacity of 86,000 Watt, with an internet capacity of 300 Mbps.

Evaluation of process aspect
The results of the evaluation on the process aspect are in the form of learning requirements data obtained through interviews with the principal, vice principal, and teachers in the field of mathematics studies, as well as strengthened data on related documents. The number of students per class at SMPIT Al-Mumtaz is 25 people, while the standard number for SMP/MTs is 30 people per class. Based on these data, the group has met the standard. Furthermore, the teacher's workload includes the main activities of planning, implementing, and evaluating learning. The planning in question refers to prepare the annual program, semester program, time allocation, syllabus, and lesson plans. Furthermore, textbooks on mathematics studies have been adequate at SMPIT Al-Mumtaz. Based on the explanation above, the researcher concludes that the learning requirements have been sufficient so that the evaluation stages of the process aspect were categorized good.

Evaluation of product aspect
Evaluation of product aspects in the online learning program at SMPIT Al-Mumtaz includes the achievement of student learning outcomes at SMPIT Al-Mumtaz at mid-term and final exam. Based on the results of the evaluation on the product aspect, the learning outcomes of students who reached the minimum completeness criteria standard of 75 on average are 90% (Figure 2). These results indicate that the online learning program in mathematics studies at SMPIT Al-Mumtaz is quite good when viewed from the learning outcomes of students in online learning. Meanwhile, the achievement of minimum completeness criteria standards in face-to-face learning is an average of 80%.

DISCUSSIONS
The online learning implemented at SMPIT Al-Mumtaz received various views from diverse parties, including: teachers, parents, and students. Most of them think that online learning has many challenges to overcome. However, in overall they argued that online learning has some benefits. One of the benefits of the online learning process is to improve the quality of education in Indonesia (Samoling et al., 2021). The success of online learning carried out by an educational institution must be measured and evaluated. Online learning requires qualitative and quantitative measurements to determine its success (Özhan & Kocadere, 2020). Learning mathematics online is an alternative that can be applied to the current pandemic conditions. The implementation requires planning, preparation, and improvement of all elements that apply online learning activities in mathematics (Ardiyanti et al., 2020). This research evaluated mathematics online learning using the CIPP Model. The purpose of the CIPP model is to make it easier for evaluators to carry out evaluation activities that include four aspects: context, input, process, and product. It should be understood that the ultimate goal in evaluation is to improve, not prove (Stufflebeam, 2003). According to Asadi et al. (2016), context evaluation examines essential issues, including school curricula and textbook assessment. In this study, the context was evaluated through document analysis which revealed that the utmost goal of the school system is to provide high quality education that engages students so that it can be well designed. Another main objective focuses on character development in accordance with SMPIT's vision and mission, namely to enable them to meet the challenges of an everevolving competitive world.
According to Stufflebeam (1971), input evaluation provides data in the form of information about the advantages and disadvantages of the selected and structured approach to achieve the given objectives. Evaluation of inputs in this study includes the availability of resources to achieve goals and fulfill needs. The findings show that the school system of SMPIT Al-Mumtaz Pontianak has a balanced curriculum. In addition, the main purpose of the input evaluation is to help students analyze alternatives related to the needs of a program. In the evaluation of the input process, the researchers decide how the activities will work to achieve the stated program objectives (Basaran et al., 2021).The core purpose of process evaluation is to describe those related to all activities in the program (Stufflebeam, 1971). When it comes to schools, process evaluation refers to the types of activities carried out in the planning phase of the instructional. This study also evaluates and provides descriptions of different teaching and learning processes.
The research findings show the details of the varied processes that help in teaching and learning. According to the principal and teachers, various types of approaches are applied to schools that make the process effective, including student-oriented activities.Product evaluation assesses the results and outputs and the unwanted, in this case, the expected and unexpected learning outcomes. This stage is to determine decisions regarding the continuation of a program and describe the impact based on the evaluation (Stufflebeam, 2003). This study includes the evaluation of school products to assess whether the objectives meet the targets. The research findings show that effective environment, relevant content, appropriate and effective teaching and learning, and the use of different teaching and learning strategies had a great impact on the students' skills, attitudes, behaviors, values, and outcomes. The question "Did it succeed?" arises at this stage because this question focuses on the product evaluation aspect (Doyok, 2021). The above questions are answered by depicting student learning outcomes which tend to increase in the field of mathematics when learning online compared to face-to-face learning, although the increase is not significant.
The CIPP model is considered capable of being part of the success of a program being implemented. This is in line with the results of research by Bhakti (2017), Fahruddin (2020), and Riptiani et al. (2015) that stated that the results of the evaluation of the CIPP model can increase the level of optimal accuracy as an evaluation material. On the contrary, Mirzazadeh et al. (2016) found that the CIPP model failed to answer some of the evaluation questions and resulted in an evaluation burden. Therefore, the evaluation that has been carried out should continue to be evaluated. This is in line with Stufflebeam (2011) that good evaluation requires the evaluation effort itself to be evaluated.Based on the research's findings, the researcher hopes that in the future, before carrying out evaluation activities, an evaluator must holistically understand the evaluation model that is applied when evaluating a program so that the implementation process is right on target. Learning mathematics online requires careful planning so that it becomes an effective and efficient learning. In addition to careful planning, monitoring and evaluation must also be carried out in its implementation.
The CIPP model has the potential to improve the quality of a program by referring to the dimensions of context, input, process, and product. As for the context dimension, it is necessary to pay attention to how the planned online learning program supports the vision and mission of an educational institution, in this case, SMPIT Al-Mumtaz. In the input aspect, the researcher recommends comprehensively completing the existing data, not only qualitative data but also strengthened by quantitative data. In the process aspect, the researcher recommends considering the teacher's workload in the online learning implementation. A workload that exceeds the limit of a teacher will affect the quality of his teaching. Meanwhile, in the product aspect, oriented to the results of online mathematics learning outcomes, researchers and evaluators can provide alternative decisions regarding the sustainability of online mathematics learning.

CONCLUSION
The implementation of the learning was quite effective, including the workload of teachers, the number of study groups, as well as a complete and well-organized class management. However, the findings reveal that what needs to be evaluated is the less optimal learning time management in order to achieve the learning objectives. In addition, the assessment stage, which is part of the learning process, was not right on target.The success of online learning in Mathematics is determined by the planning and readiness of the teacher and other supporting tools for success. Therefore, it is essential to hold an activity in order to increase teacher's understanding on online learning.