DEVELOPING ASSESSMENT AS LEARNING BASED ON PROBLEM-SOLVING TO PROMOTE UNIVERSITY STUDENTS ' SKILLS IN CRITICAL-CREATIVE THINKING

Keyword: Assessment as learning, critical thinking, creative thinking, problem-solving Assessment as learning is an assessment to guide and provide opportunities for participants to monitor and reflect critically on their learning processes and outcomes. This study aimed to produce assessments as learning based on problem-solving that could be used to improve university students' skills in critical-creative thinking and to test the validity, reliability, and effectiveness of the assessment. The research design used was a 4D model, which stands for 'defining, designing and developing, and disseminating. The research data consisted of two categories, which were qualitative and quantitative. The qualitative data were in the form of comments, criticisms, and suggestions from the expert team specializing in assessment; the quantitative data were in the form of critical-creative skills scores of product users. Qualitative data sources were the expert assessment team, and quantitative data sources were the fourthsemester college students of the Indonesian Department, Faculty of Letters, Universitas Negeri Malang. The research findings show that the product was valid and reliable. Besides, the result of the test of effectiveness indicated that this assessment could improve university students' skills in critical-creative thinking.


INTRODUCTION
Assessment is a critical part of effective teaching (Griffin, P. & Care, E., 2015).The new vision for assessment states that assessment cannot be separated from learning because the results of the assessment can be used to provide feedback on the learning process and results (Earl, L. M., 2003).Assessment activities are used to see the extent to which instructional objectives have been mastered by the learners in the form of learning outcomes that are shown after they carry out teaching and learning activities (Harsiati, 2017).Assessment as a process has three main functions, namely measuring progress, supporting the preparation of learning plans, and improving or refining learning processes and results (Griffin, P. & Care, E., 2015).The purpose of the assessment is to determine students' understanding before starting learning, to check the development of understanding during learning, and to obtain the learning outcomes achieved at the end of learning (Wragg, E.C., 2001).Unfortunately, the assessment function as a process is ignored because most of the types of assessments carried out in higher education are the assessment of learning, an assessment carried out after the completion of learning.Midterm exams and final exams are examples of forms of assessment of learning.
Many countries have begun to review and rethink the form and function of assessment (Earl, L. M., 2003).They began to discuss the use of assessment as learning to optimize learning outcomes.Assessment as learning emphasizes using judgment as a process to develop and support learner metacognition (Earl, L. M., 2003).Assessment as learning focuses on the role of students as critical thinkers and the link between their assessment and learning.Students act as critical thinkers, understand information, connect it with prior knowledge, and use it to build new learning.
Assessment as learning is a type of formative assessment whose purpose is to make descriptions, notes for improvement of the process and subsequent learning outcomes (Koc, S., Liu, X., & Wachira, P., 2015).If assessment as learning is used in the learning process in universities, the lecturers can collect a wide range of data for a different purpose -so that they can modify the learning work for their students.Lecturers can use the learning process data to find out which aspects are understood and not yet by the learner.Based on the process data, the lecturer can design improvements in the learning process in the next stage.Process data can also be used by lecturers to improve learning skills in creative-critical thinking.
Critical-creative thinking is the combination of two important skills needed to survive and thrive in the 21st century.The ability to think that involves analyzing and evaluating information, ideas or situations is categorized as the domain of critical thinking skills and the ability to find innovative new ways to solve problems, answer questions or express meaning through analysis, synthesis or repurposing knowledge categorized as creative thinking skills (Forum, 2015).Critical and creative thinking are both achievements of thought (Marzano, R. J. dkk., 1988).The process of assessing or judging is the domain of critical thinking while making or producing is the domain of creativity (Paul & Elder, 2008).These valuable skills help pupils to face various challenges in life effectively, such as solving their daily problems and problems at the workplace in the future.Both of these skills are important and need to be trained among pupils and also college students.
The critical-creative thinking can be improved through a combination of assessment as learning and problem-solving based assessment.Assessment as learning that focuses on process assessment can be used to track the development and quality of learning, containing tasks that require students to find problems, analyze and evaluate them, and then work out the solutions (Jonassen, 2010).Assessment as Learning Based on Problem-Solving is very suitable for use in universities (Gok, T., 2010).
Based on the above background, this study aimed to produce assessments as learning based on problem-solving that could be used to improve university students' skills in critical-creative thinking and to test the validity, reliability, and effectiveness of the assessment.

THEORETICAL FRAMEWORK
This section describes the nature of the assessment of, for, and as learning, critical-creative thinking, and problem-solving assessment.

Assessment of, for, and as learning
There are three types of approaches in assessment, namely: assessment of learning, assessment for learning, and assessment as learning (Earl, L. M., 2003).Assessment of learning is usually used as a form of institutional accountability to students and parents related to the achievement of student learning outcomes.This type of assessment is carried out separately from teaching and learning activities, meaning that the assessment is carried out at a separate time, outside the learning process.The middle and end of semester examinations are examples of assessment of learning.This is different from assessment for and as learning which implementation is integrated with the learning process.Formative exams are examples of assessment as and for learning.Assessment for learning is carried out by the teacher, while the assessment of learning is carried out by students.Self-assessment and peer assessment are examples of assessment as learning.In assessment as learning, students can also be involved in formulating assessment procedures, criteria, and assessment rubrics/guidelines so that they exactly know what to do in order to obtain maximum learning outcomes.
The difference of assessment of, for, and as learning can be seen in Table 1 below.

Student
From table 1 above, it can be concluded that the three types of assessment have their own place and function, but what needs to be considered is the trick to get the balance right.During this time, assessment of learning is predominantly conducted by educators in universities, compared to assessment for learning and assessment as learning (Earl, L. M., 2003).Assessment of the achievement of learning outcomes should prioritize assessment as learning and assessment for learning rather than the assessment of learning.The following is an ideal assessment pyramid.
Figure 1 Reconfigured Assessment Pyramid (Earl, L. M., 2003) More than 4.000 studies show that when formative assessment (as and for learning) is carried out well in class, it can increase the speed of college student learning because a process that works well can generate huge benefits for student achievement, and even though it is used by different teachers in various ways, but still gets extraordinary results for students (Popham, J, 2011).The idea of formative assessment is important to develop the skills needed to live in the 21 st century.This assessment aims to equip lecturers with a variety of information to college students in promoting and developing 21st-century skills.(Griffin, P. & Care, E., 2015).
This study developed assessment as learning.The assessment function as learning is to provide descriptive and accurate feedback for each learner so that it will help to develop independent learning habits and detect the improvement in critical-creative thinking skills.In addition, in this assessment learners are given the opportunity to reflect on ideas to adjust, rethink, and articulate their learning processes and outcomes (Black, P., Harrison, C., Lee, C., Marshall, B., & William, D., 2003).

Critical and creative thinking
Critical thinking involves an ability related to assessing, judging, or evaluating a given problem or topic, as well as evaluating your own reasoning (DeWaelsche, S.A., 2015).Critical thinking is reasonable and reflective thinking focused on deciding what to believe or do (Costa, 1991).Being reasonable means showing a reason or sound judgment, and being reflective means to think seriously about what we have read or heard and involves any activity to evaluate or assess the validity of the information and opinions we encounter.Critical thinking involves bringing outside knowledge and values to deliver and evaluate the presentation and decide what to ultimately accept as true (Facione, P. A., 2015).
Critical thinking skills are needed to validate information, synthesize information, communicate information, collaborate information, and solve information in the digital age so much and come from various sources that information is not necessarily reliable.Critical thinking skills are needed so that we can survive and even play a role in life in the 21 st century.
Creative thinking is thinking patterned that tends to lead to creative results and focused more on the product rather than the process (Marzano, RJ et al., 1988) and the creative result is being novel and useful (Harrington, 2018); (Gaut, B., 2010).Being a novelty refers to the idea that the product of creativity has to be something "different, new, or innovative".The useful reference to the creative 'thing' has to make sense or be useful for a particular context.Also, creativity is strongly associated with imagination, innovation, originality, and genius (Pope, 2005); (Swann, Pope, & Carter, 2011).
Creative thinking is detected in four forms, namely sensitivity, fluency, flexibility, and originality.In connection with sensitivity, authenticity, flexibility, and fluency in the thinking process that gave birth to ideas (creative) it was deemed necessary to have a further action to correct and organize well or regularly and in detail what was produced.This case needs to be done so that the individual does not lose momentum in the learning atmosphere, especially before he can forget the good ideas that emerge.This orderly and detailed arrangement opens the opportunity for him to be able to repeat or read and review what he produces at any time.

Problem-solving-based assessment
To thrive in the 21 st century, students need more than traditional academic learning.They must be adept at collaboration, communication, and problem-solving (Forum, 2015).The word problem has many meanings.The problem is defined as a state to be solved.It is also interpreted as is an unknown solution.
If we have a goal and do not know how to reach that goal, that is an unknown solution (Jonassen, 2010).The problem is defined as a situation without a clear solution.There are two important attributes to define problems, namely (1) problem is an unknown or previously unknown entity (2) to solve the unknown, we must use social, cultural or intellectual values (Jonassen, 2010).
Whereas, if the problem is an unknown entity, then the problem-solving is cognitive operation directed to find that unknown entity (Anderson, 2010).Cognitive operation in problem-solving requires a mental representation of the problem, and its context is successful.Problem-solving requires learners to actively manipulate and test their models (Jonassen, 2010).A successful problem-solving and try out solutions in their minds (mental models or problem spaces) before trying them out in the physical world.Most of the contemporary research and theories in problem-solving claims that problem-solving skills are specific domains and contexts, that is the problem-solving activities are located, embedded, and therefore dependent on the nature of context or domain knowledge.
From the level of complexity, problems can be sorted into two types, namely simple problems and complicated problems (Jonassen, 2010).A simple problem is in small-scale, disconnected from other problems, has no great consequences, the solution is not too complicated, and can be solved individually.The extent of the problem is limited to individual and can be solved by an individual as well.The complex problems have large scopes, can be related to many other issues, have enormous consequences, and the solution requires group collaboration and in-depth analysis.The range of complex problems is related to many individuals and can be solved by many individuals as well.
From the stimulus, the problem can be classified into eleven types namely logical problems, algorithmic problems, story problems, rule-using problems, decision-making, troubleshooting, solutionproblem diagnoses, tactical/strategic performance, case/problem analysis systems, design problems, and dilemmas (Jonassen, 2010).The types of problems are categorized as well-structured problems and illstructured problems.The characteristics of well-structured problems are presenting all elements of the problems, requiring convergent answers, prescribing solution processes; ill-structured problems often requiring personal opinions or beliefs about the problems.
Problem-solving is an additional assessment of 2003, 2012 domain in Program for International Student Assessment (PISA).This research has led to advances in understanding and measuring individuals' problem-solving capabilities (OECD, 2014).In PISA 2003, static tasks included decisionmaking problems, where the student has to choose among alternatives under constraints, and systemanalysis problems, where the student needs to identify relationships between parts of a system.PISA 2012 also used creative problem-solving assessment to assess individual ability to solve real-life problems of a general nature.The problem-solving assessment in PISA 2012 focused on general cognitive processes involved in problem-solving, rather than on the ability to solve problems in particular school subjects.
Furthermore, a problem-solving-based assessment requires students to respond critically as well as simultaneously to construct a product rather than select from a set of predefined alternatives or answers.Therefore, this assessment as learning based on problem-solving is important to be developed to strengthen or replace multiple-choice assessments that only select the already provided answers.

METHODS
This study uses a development research design adapted from the Recursive, Reflective Design and Development model or abbreviated R2D2 (Willis, J., 1995).In the R2D2 model, there are three development focuses namely (1) define (2) design and development, and (3) dissemination (Willis, J., 1995).Of the three focuses, only two focuses were chosen, namely define and design and development.There are two main activities carried out on define focus, namely (1) the establishment of developed product specifications and (2) the establishment of participatory teams.There are two main activities carried out on the focus of determining design and development, namely (1) preparation and (2) development.
In development activities, a product tryout, test validity, and reliability are carried out.Product tryouts are carried out to get feedback.Feedback is used to improve the product that has been developed (Willis, J., 1995).Feedback in the form of criticism, comments, and suggestions.In development activities, tryout product was carried out, namely from the validity and reliability test.Tryout of the product to get feedback.Feedback is used to improve the product that has been developed (Willis, J., 1995).Feedback in the form of criticism, comments, and suggestions.
For the purpose of testing the effectiveness of the product, a pre-experimental design was used, namely a single group pretest and posttest design.

FINDINGS
This section describes the research results, including the description of the main features of the assessment as learning based on problem-solving to improve university students' skills in critical-creative thinking, the validity, reliability of the test, and the effectiveness of the assessment.The descriptions can be found at the followings.

The main features of assessment as learning based on problem-solving
The product developed in this research is a problem-solving assessment model used to promote the ability of college student in critical-creative thinking.The assessments contain test models, scoring guidelines, and the syntax of the assessment as learning based on problem-solving.The objectives of the test focus on six skills namely (1) identifying a problem, (2) conceiving solution strategies, (3) setting learning task, (4) examining the best solution, ( 5) designing text framework to the best solution, and (6) proposing a solution in the form of creative writing text.The type of test was an essay-story problemsolving.
The material stimulus used to develop essay tests originated from news texts and exposition texts that address social issues so that students can provide solutions to social problems faced by the environment/nation.The examples of social problems are the rise of hate speech in social media, the clash of supporters, rampant fraud, early marriage, or drug abuse.The writing style of the material stimulus is narrative and argumentative.The length of the text is 900-1200 words.
The main features of the assessment as learning based on university students' skills in criticalcreative thinking problem-solving to promote the university are presented in The main features of the assessment as learning based on problem-solving shown in Table 2 were used to develop the blueprint of the test and also as a reference for developing the test.In addition to the tests, the scoring guideline is also developed.The score is ranged from level 0-4, the best level is 4 and the worst score is 0. There are six major sub-skills assessed, which are the ability to identifying a problem, conceiving solution strategies, setting learning task, examining the best solution, designing text framework to the best solution, and proposing a solution in the form of creative writing text.The full scoring guidelines are listed in Table 3 below.

The Syntax of Assessment as a learning
Assessment as learning in this study was developed to facilitate college students to promote critical-creative thinking skills integrated with the learning process to improve the learning process and stimulate further learning.The following is an example of the learning syntax for carrying out assessments as problem-solving-based learning in teaching Reading for units 1-2 nd and 3 rd meetings.on the results of their works.12. Learners revised their work based on the results of the self-assessment and peer assessment.13.Learners describe the learning strategies that have been implemented and analyzes their weaknesses and strengths.14.Learners are asked to rethink the acquisition of learning outcomes and think about improving the appropriate learning strategy.

The validity of the test
The expert judgment on construct and content test validity indicated that assessment as learning based on problem-solving was valid because both expert 1 and expert 2 stated that all items were in line with the theoretical assessment as learning based on problem-solving.The summary of test validity results can be seen in Table 5 below.The reliability of the test The inter-rater reliability test of the essay test packages was performed by asking two teachers to score the work results of the 18 students, using the scoring guideline (See Table 3).The reliability test results had a high and positive correlation value, which was 0.843 with a significance level of 0.000.The correlation score indicated that the correlation score between teacher 1 and teacher 2 was significant and the positive value indicated that the pattern of the correlation score between teacher 1 and teacher 2 had the same direction.When teacher 1 gave a high score, teacher 2 also gave a high score.This case means that the essay test is reliable.The summary of the reliability test results can be seen in Table 6 below.

Product effectiveness
From the results of different test statistics on the results of creative-critical thinking skills based on problem-solving before treatment and after treatment it is known t = -5.826and P = 0.000.Thus, there is a significant difference between reading ability scores before and after treatment because P <α is 0.05.The treatment is the use of assessment as learning based on problem-solving in reading lectures.So, teaching reading with the use of assessment as learning based on problem-solving significantly influences critical-creative thinking.This case means that this problem-based assessment as learning can improve university students' critical-creative thinking skills.

DISCUSSIONS
Based on the results of the validity test, it can be concluded that the assessment developed in this study is valid to be used to promote critical-creative thinking skills.This case is because the assessment involves six cognitive operations which are (1) identifying a problem, (2) conceiving solution strategies, (3) learning task settings, (4) examining the best solution, ( 5) designing text framework to the best solution, and (6) proposing a solution in the form of creative writing text.Identifying the problem based on the fact of the scenario (story problem-solving) including potential causes is a critical reading activity.Activities involved in the first activity are understanding the story problem-solving, identifying the problem contained in it, analyzing it, and then determining the potential causes of the problem.This case is in harmony with the critical thinking feature that involves bringing knowledge and values to deliver and evaluating what is truly accepted as true (Facione, P. A., 2015).The essence of critical reading activity is an activity to analyze and evaluate what we read, working out what is good and what is not, and why.
Proposing a number of solution strategies and explaining the development process, and connecting with the learner's experience, and proposing a solution in the form of creative writing text are some forms of creative thinking skills.This case is because college students do not only understand the text but also respond to the text content.Students are also asked to express the best solution in essay form.The essay contains at least four or five full paragraphs containing one introductory paragraph, two or three paragraphs detailing the original ideas offered for problem-solving, and a closing paragraph.This case is in line with the characteristics of creativity associated with imagination, innovation, originality, and generosity (Pope, 2005).Creative thinking is thinking patterned that tends to lead to creative results and focus more on the product rather than the process (Marzano, R. J. et al., 1988), and the creative result characteristics are being a novelty and useful or valuable (Harrington, 2018); Gaut, B., 2010).
At the implementation stage of the assessment as a learning tool, educators provide opportunities for learners to reflect on their activities and learning outcomes.In addition, learners also conduct selfassessment and peer-assessment, providing constructive assessments and feedback on the process and the result of their and their friends' work.It is based on the belief that students are seen as subjects who are able to judge themselves and each other (peers).Assessment as a learning tool is needed to remember the following points.Student self-assessment is guided by a self-assessment rubric.Students are given a selfassessment rubric that contains detailed indicators of competencies that have been/have not been mastered.Assessment can also be done using a self-reflection sheet or journal.Students fill out a selfreflection sheet or journal every time they finish learning a competency.
Assessment for learning has the function of guiding and providing opportunities for each student to monitor and critically reflect on their own learning activities and determine the next learning step or strategy.The metacognitive formation can be done by giving students the opportunity to critically reflect on their learning steps and strategies.By having good metacognitive abilities, students get descriptive and accurate feedback that can help or develop independent learning habits (Bailey, A. L. & Heritage, M., 2008).

CONCLUSIONS AND SUGGESTIONS
From the above description, it can be concluded that the assessments as learning based on problem-solving were valid as seen from the construct and content validity.This case means that this assessment can be used to promote college students' critical-creative thinking skills.This assessment is also reliable which means this assessment can produce trusted or reliable measurement results.
Based on the results, it is recommended to implement these assessments as learning based on critical problem-solving, creative reading assessments based on problem-solving as one of the formative assessments.This formative assessment has the following benefits (a) facilitating students to assess, rethink, and demonstrate their learning activities, (b) encouraging students to think critically and creatively, (c) improving student collaboration and interpersonal communication skills, (d) encouraging students to learn from each other (from peers), (e) creating conditions so that learners and educators can discuss alternative solutions.
Experimental design can be seen in Diagram 1 below.O1: pretest score O2: posttest score (learning outcomes) X: Assessment as learning based on problem-solving Diagram 1: Single Group Pre and Post-test DesignThe effectiveness of the product was tested with 35 college students of the Indonesian Department, Faculty of Letters, Universitas Negeri Malang who had passed basic reading courses.Product effectiveness testing was done to see the effectiveness of the product in terms of learning outcomes.This research data consist of two categories namely quantitative and qualitative data.The quantitative data were in the forms of the expert team's questionnaire results, the score of reliability test of the fourth-semester college students of the Indonesian Department, Faculty of Letters, State University of Malang.The qualitative data were in the validity of the contents and constructs of instruments from expert teams.The sources of research data were two experts who were a doctoral degree qualification.A reliability test was tested to 18 eleven graders.The research instruments in the questionnaire for product validity test and the form of test complete with scoring guidance for reliability tests.

Table 3
Scoring guideline of the assessment as learning based on problem-solving Dimens ions

Table 6
Summary of test reliability result of the critical-creative reading assessment based on problem-solving