College Students’ Perceptions of an Automated Writing Evaluation as a Supplementary Feedback Tool in a Writing Class

Automated Writing Evaluation (AWE) programs have emerged as the latest trend in EFL writing classes. AWE programs supplement teacher feedback and offer automated sugges- tions and corrections to students' linguistic errors such as grammar, vocabulary, or mechanics. As there is a need for better recognition of different AWE brands utilized for different levels of students, this research sheds light on identifying six university students’ views of an AWE program, namely ProWritingAid (PWA). The six students are categorized as having high or low writing achievement. This descriptive study delineates the students’ perceptions qualitatively. A semi-structured interview was used to collect the data. The findings suggest the students’ positive views of PWA because it could make class time more effective; it had useful feedback on grammar, vocabulary choices, and mechanics, building students ‘self-confidence over their compositions. In addition, for different reasons, the students engaged differently with PWA to enhance their drafts, e.g., using PWA only for the first drafts or for the first and

Nonetheless, many teachers and researchers also perceive AWE unfavorably. Many teachers also overlook the implementation of AWE in their writing classes. Most of the reasons are related to the teacher's skill in using a computer or some defects found in the programs' features that cannot fully describe the students' errors (Chen & Cheng, 2008). Moreover, AWE is claimed to dehumanize the writing class for removing the human aspect (Warschauer & Ware, 2006) or by limiting the meaningful interaction in the writing process (Wang et al., 2013;Fauzi et al., 2020).
Conclusively, the adverse claims of AWE become a consideration as to how teachers, students, or researchers should use this program. The first consideration is the readiness of the teacher and students to use the program (Cotos, 2010), which also implies the teacher's and students' skills in using technology. Indeed, teachers' and students' digital skills can affect their perceptions of the automated feedback program (Bakla, 2020). Hence, it can directly affect the implementation of the strategy (Chen & Cheng, 2008). The second consideration is the students' level. It has been reported that not all students need this program, particularly students with a high-A R T I C L E I N F O A B S T R A C T proficiency level since they need more input on the content of their compositions rather than input on grammar or mechanics (Chen & Cheng, 2008).
Along with the level of the students, the drawbacks of the program also another thing to be considered. Indeed, AWE is limited to evaluate the semantic interpretation of a text, in other words, the content of a text. Subsequently, AWE is not recommended to be used by disregarding teacher feedback (Crusan, 2015;Wang, 2015;Wilson & Czik, 2016). Granted that the teacher feedback is still present, AWE is not used to dehumanize the class or substituting teacherstudent interaction since the teacher's role in providing feedback is still there.
Therefore, the issue that should arise is not how to use AWE solely without teacher feedback, but how the implementation of AWE and teacher feedback can benefit the students and teachers, as well as how students or teachers perceive such implementation. However, many studies about AWE appear to focus on comparing many writing classes with or without AWE in class (Dikli & Bleyle, 2014;Yinghui & Dan, 2015). As a result, the students' perceptions are less discussed, especially the EFL non-English major students. Most AWE-related studies have mainly been conducted among the first, second, or foreign English language students, most of whom are English-major students (Chen & Cheng, 2008;Lu, 2019;Parra & Calero, 2019;Yinghui & Dan, 2015). Thus, it is suggested to perform the AWE in non-English major students (Yinghui & Dan, 2015). It has also been suggested to find out how the implementation of a different AWE program is perceived by the students (Hegelheimer et al., 2016). Certainly, the interface of different AWE programs can be one factor that determines its impact to the students. Therefore, it is proposed that different AWE programs should be investigated to identify their potentials (Stevenson & Phakiti, 2014).
To this matter, considering the conditions aforementioned above, we had investigated a five meeting class using the combined feedback of teacher and an AWE program in one ESP (English for Specific Purposes) paragraph writing class (Ariyanto et al., 2019). The strategy involved the use of teacher feedback and a relatively new AWE free program called ProWritingAid (PWA). Nonetheless, this study specifies the students with high and low writing performances only since numerous studies have indicated that different writing skills are one factor that affects the success of AWE implementation (Chen & Cheng, 2008;Warschauer & Grimes, 2008;Zhang & Hyland, 2018). Our study also found a claim from the teacher that low English proficiency students in this class may not be benefited from the program (Ariyanto et al., 2019). As a result, the research questions raised are: • How do students who have high and low writing achievements perceive the need of PWA as a supplementary to teacher feedback to improve their drafts?
• How do students who have high and low writing achievements utilize PWA to improve their drafts?
A. Teacher Feedback Teacher feedback is an essential key in the writing process. Teacher feedback has been identified as very beneficial to improve the students' writing performances (Muchemwa et al., 2019) and as a characteristic of a successful teacher in various subjects and classes (Gan et al., 2018). Teacher feedback can be given in various forms such as direct or indirect, focus or unfocused, as well as oral or written, etc. The complexity of the students' needs and their writing skills determines the type of feedback the students' needs (Lee, 2019). However, effective feedback from teachers has been seen as the most valuable feedback. Many students cannot achieve the maximum benefit of teacher feedback when teachers often offer error codes unaccompanied by any description (Muchemwa et al., 2019). Previous studies have also shown that many students do not have a favorable view of the code or general remarks made by the teacher (Ferguson, 2011).
Improving the students' writing skills, thus, requires not only a certain form of feedback but also effective feedback. Feedback is effective if it is frequent, sufficient, comprehensive, and understandable and linked to the objective of the assessment (Glover & Brown, 2006). However, it can be hard for a teacher to give the same amount of feedback to students (Montgomery & Baker, 2007). Teachers' perceptions of the importance of feedback are often in contrast to their practices (Li & De Luca, 2014). Many teachers also feel that giving teacher feedback is time consuming and exhausting (Zupanc & Bosnić, 2015). Technology developers invented the AWE program that can bridge the gap and help the teacher reduce the teacher's workload and provide effective feedback to students' linguistic errors.

B. Combination Feedback from Teacher and AWE
Program AWE can enhance students' writing quality concerning its accuracy in syntactic analysis. According to (Aken, 2017), the syntactic analysis method in a program can improve writing structure and address elements of style in writing such as grammar, spelling, punctuation, voice, and others. AWE also offers descriptive feedback or more detailed infor- mation on a variety of errors (Zupanc & Bosnić, 2015). Unlike the above-mentioned issues with teacher feedback, AWE offers an error correction code and an overview that can help students learn about their errors. Nonetheless, since its precision mainly relates to syntactic analysis, it has not yet been able to provide feedback on the content of a text.
In fact, the combination of AWE and teacher feedback is suggested due to the limitation of AWE as well. AWE is confined to semantic language processing. None of the programs can assess the verity of context or facts written in a text (Lu, 2019;Zupanc & Bosnić, 2015). Therefore, teacher feedback can be given to evaluate the text organization and content when AWE is used (Crusan, 2015). AWE can be used to check any errors as it offers immediate error evaluation, thereby reducing the teacher's workload (Wilson & Czik, 2016). It is also claimed as an effective feedback agent when it is utilized as an additional tool to support teachers instead of replacing teachers (Cotos, 2010). As an example, it can be used with the student's earlier draft before submitting it to the teacher so that teacher can focus on the other aspects of writing (Wang, 2015).
One study that exclusively explores the effectiveness of using this combined feedback is Wilson & Czik (2016). This research finds support for the claim that using AWE with teacher feedback can increase the students' writing quality. Earlier research which investigates a naturalistic classroombased inquiry of AWE in class find many students who favored AWE if it is used to help students improve their linguistic accuracy. At the same time, the teacher focused on helping the students to enhance the meaning negotiation and idea development in the students' writings (Chen & Cheng, 2008). With promising findings, there have been many suggestions for implementing this program in writing class.

II. Method
This research is a descriptive study to investigate six ESP (English for Specific Purposes) university students' perceptions of ProWritingAid implementation, specifically those with high and low writing achievement. To address the research questions, a semi-structured interview with six students was carried out to define the students' perceptions qualitatively. A document review on the students' achievement result was done to identify further and see the match or mismatch between the students' perceptions and their performances. To ensure the anonymity of the participants, a code was assigned to the students, for example, S1, S2, & S3 refers in this study were those students with a high writing achievement, whereas S4, S5, & S6 were those with low writing achievement.

A. Participants
Informatics Engineering was the major of the six students taking part in this study. During the implementation of PWA, the students were taking an ESP Writing course (paragraph writing) in their second semester. These students were chosen from a total of 33 students. S1, S2, and S3, who got high writing achievement, had a score above 95 in their descriptive paragraph test. They were selected randomly from other students with scores between 90 to 100. Meanwhile, S4, S5, and S6 had a score below 71; also selected randomly from other students with scores between 70 to 60. The teacher suggested the score range, considering how the score represents the English level of the students. See Table 1 to refer to the students' scores.
The students' achievement results were taken from the achievement test (writing a descriptive paragraph) at the last meeting. The teacher had scored the students' paragraphs using the teacher's rubric and assessed them according to 5 writing criteria. The five criteria had different weighting scales each such as organization (20%), content (30%), grammar (20%), mechanics (15%), and vocabulary (15%). The scoring rubric indicated 4 levels points for each criterion, thus the minimum point was 1 and the maximum was 4.

B. ProWritingAid
ProWritingAid (PWA) free version is a webbased AWE program to evaluate a text (see Figure  1). People can access this program online at https://prowritingaid.com/. PWA has various features such as style, structure, or sticky sentence correction features. However, as far as we know, only a few studies have discussed this software to be used in a classroom setting. Thus, it is a motivation to do this study to fill the gap. It is also worth noting that any company, ProWritingAid, does not sponsor this study.

C. The Utilization of ProWritingAid as Supplementary to Teacher Feedback
The implementation of PWA was done in five weeks or five meetings. During those meetings, the feedback process ( Fig. 2) was repeated three times for three writing assignments. Before the implementation of the strategy, the teacher was trained to use and experience the program. PWA was used to help students fix errors related to grammar, vocabulary choice, sticky sentence, and mechanics (punctuation, capitalization, and spelling) identified in the students' paragraphs. In the meantime, teacher feedback emphasized the content and organizational aspects of the students' paragraphs, but not restricted to linguistic errors as well. Figure 2 demonstrates the teacher's strategy to use PWA.
There were six stages applied to the students' composition after the students had wrote their first draft. Stage 1 until 3 were done outside of the class hour. Meanwhile, the students' drafts that contain teacher feedback were always handed back during the class hour. Before the students revised their drafts, the teacher always had a review session presenting the students' most common errors, undetected errors by PWA, and the expected outcome that students ought to be aware of. Then, the students revised their paragraphs according to the teacher feedback (Draft 3) at home. Some of the students submitted their third drafts as their final drafts, while others used PWA to correct their third drafts again and submitted the fourth draft. However, for Assignment 3, stage 1 was ignored as the students wrote their paragraphs in class. Finally, the teacher gave a paragraph writing test in the last meeting.

D. Data Collection and Analysis
The data in this study (student opinion) was obtained following the implementation of PWA and teacher feedback. A semi-structured interview in the students' native language was employed to obtain the data from each student. The data from the interview were transcribed and later translated. It was then analyzed as QDA or Qualitative Data Analysis. Guided by the research questions, a theme analysis of the transcriptions was performed. Firstly, the transcription was coded. The coding process is purposefully done to identify the data. Secondly, the data was generated in themes to find some patterns and congruities. A theme means a group of various codes in similar categories or characteristics (Vaismoradi et al., 2013). Thus, the purpose of making the themes is to provide meaning from the dataset concerning the research questions. The identified data to answer the research question were analyzed and the categories were refined. This way, the researcher ensures that the identified perceptions or reasons also resonates with the transcriptions. Finally, some representative quotes from the transcriptions were selected, presented, and discussed. In addition, the students' achievement result from the teacher was used to assist or contradict the students' claim.

A. The Needs of PWA Feedback to Improve the Students' Drafts
Based on the data analysis, the main themes from the first research question, "How do students with high and low writing achievements perceive the need of PWA as a supplementary to teacher feedback to improve their drafts?" were organized (e.g., effective review session, useful feedback, and self-confidence builder). A total of five topics that shows the students' reasons of needing PWA to improve their drafts are as follows: • PWA implementation made class time more effective with review session instead of individual feedback.
• PWA feedback on grammar was useful for both high and low achieving students.
• PWA feedback on vocabulary choice was useful for both high and low achieving students.
• PWA feedback on punctuation and spelling were useful for low achieving students.
• PWA was needed as a confidence builder tool for both high and low achieving students.

B. Effective Review Session in Class
Based on the result of the analysis, we offer the most representative quotations of the students' thoughts. Conclusively, all students felt that having PWA was needed because it changed the way the teacher provided feedback in class: S3: "Before using PWA, the teacher had an individual consultation session. This consultation session was time consuming practice for a big class with more than 30 students".
When the teacher asked the students to use PWA and gave teacher's feedback outside of class hour, the teacher only had a review session in class instead of an individual consultation session. This review session allowed the students to know more about the impact of using PWA and the expected output to be performed, both in linguistic and content aspects: S1: "PWA saved time in class since it could be done at home" … "In class, we had more time for other activities or exercises such as the teacher's presentation discussing the errors undetected by PWA or the expected content that students need to perform". S5: "Teacher's presentation was beneficial because it discussed PWA's defect as well. We could know and be careful in using PWA because of it. I also learnt more about my errors".
The needs of having PWA were valued greatly by the students because of the review session. Some students also suggested that a teacher should have this activity when using PWA as it can scaffold the students in constructing a better composition: S1: "As a suggestion, when using this program, a teacher can discuss the students' most common errors instead of individual consultation because most errors will be corrected by PWA already". S6: "Although PWA helped the students so much, a teacher still needs to correct the students' paragraphs and discussed the students' error. Students have different skills, hence, the review session that discusses the students' errors is recommended to any teacher using this program".
The data analysis found for this theme shows that different students possess the same need for PWA and the review session as all students unanimously vote for it. The second until the fourth points found from the data analysis was linked to PWA's strength to provide various feedback. Firstly, all students agreed that grammar feedback was useful and gave more impact to the students' paragraphs because not only that they can correct it but also learn from it: S1: "The benefit of PWA that had the biggest impact on my writing was grammar feedback. By using PWA, I could know and correct my grammar errors by myself". S4: "My English grammar is bad. With PWA, I can correct it and learn from the explanation".
Secondly, vocabulary feedback was perceived useful by many students. The feature of vocabulary feedback allowed the students to choose for appropriate vocabulary use and change the most common vocabulary used by the students: S2: "Vocabulary choice suggestion was very useful to me. I could use different vocabulary so that my sentence would not be monotone". S5: "I used the word 'very' too many times, with the suggestions given in PWA, I could improve my vocabulary".
Thirdly, low-achieving students were found to think highly about the punctuation and spelling feedback because these students still made errors in mechanics: S5: "The improvement that I felt was in punctuation and spelling as well" .... "PWA thought me where to put coma on my sentences. When I write with PWA directly, it also checked my spelling. This feature is badly needed".
On the contrary, the high-achieving students did not perceive the need of punctuation and spelling feedback as they did not make many errors in punctuation or spelling. One of the students' statement is as follow: S3: "I did not make any error in punctuation or spelling. I only need the grammar feedback. It is the most useful feedback for me".
With the availability of feedback for grammar, vocabulary, or mechanics, students valued PWA greatly. As an automated feedback tool, it could deal with the students' linguistic errors, allowing the teacher to focus more on content or organization aspect of writing: S2: "PWA can help me reduce my errors in grammar. Thus, my teacher usually only gave me feedback on my content or organization of my sentences" …. "Sometimes, I only need to change the order of my sentence to make it easy to be read or delete sentences if necessary".
Overall, from this theme, students' views only differ on the need of PWA's mechanic feedback. This feedback was valued more by low-achieving students.

D. PWA as a Self-Confidence Builder
The last point mentioned from data analysis was related to how PWA Feedback can assure the students or build the students' confidence about their works: S2: "Although sometimes I do not find any errors from PWA, it feels good to rely on and know that I still have this machine to help me before I give my works to my teacher. I can be more confident in submitting my drafts". S5: "The existence of this kind of software is needed because it assured me about my works. Although it cannot make our paragraphs free from errors, it helps in a way".
The data analysis for this theme shows that students' achievement in writing is not an indicator for needing this program as a self-confidence builder. Most students valued the implementation of this program positively because of the impact it gives to the students' confidence when submitting their drafts to the teacher. Having the students' thoughts about their needs of PWA, their engagements with PWA to improve their drafts are also highlighted.

E. Students' Engagement with PWA to Improve
Their Drafts To answer the second research question, "How do students with high and low writing achievements utilize PWA to improve their drafts?" The students were asked about their engagements with PWA. Three themes emerged from the data analysis: • High-achieving students used PWA to reduce linguistic errors for their first drafts.
• Low-achieving students used PWA for their first or third drafts due to their poor sentence construction and content.
• Students sometimes only engaged superficially with PWA feedback because they were in a rush, confused about the given feedback, or because they knew they made mistakes.
Firstly, it was found that the high-achieving student preferred to use PWA for their first drafts or before the first submission to the teacher. The students' minor errors became the main reason why the high-achieving students only use PWA for their first drafts: S1: "I used PWA on my first drafts. After the teacher gave me feedback, I mostly found correction on my grammar. For content, sometimes I just need to change the order of my sentence and add word connector. Using PWA once for every assignment is enough for me on this matter".
For students with high writing achievement, they did not see the need to use PWA again because they had a few errors that could not be detected by PWA, therefore, they would not have a problem with submitting their third drafts since the teacher had covered the correction. They also found to have no or few suggestions on their content: S2: "I used PWA on my first drafts on every assignment. After that, I rely on the teacher to help me improve my paragraphs since the teacher can give feedback on the undetected errors by PWA and the content of my works" …. "Mostly, I did not have to change my content a lot, so I did not use PWA again".
S4 also held similar belief with the highachieving students, however, S4 chose to use PWA for the first draft only because S4 needed more linguistics feedback from the teacher than PWA. He/she did not use PWA twice for every assignment because she/he knew that PWA could not fix his problems: S4: "I need teacher feedback more especially about my sentence construction" …. "In class, my teacher had ever discussed that my errors were almost categorized as those which were undetected by PWA, especially due to my poor English and grammar knowledge, thus my teacher often gave the correction directly on my drafts".
Secondly, unlike the students who had high achievement, the students with low writing achievement mostly used PWA again to reduce their errors after they had revised their works according to teacher feedback. In contrast with the high-achieving students, the low-achieving students needed to revise not only their linguistic errors but also their content. The students' remarks are as follow: Nonetheless, it was found that low-achieving students cannot be critical about the feedback given by PWA compared to the high-achieving students.
S6: "I sometimes found that the correction I had made with PWA was corrected again by my teacher. Indeed, my teacher reminds me to be careful in using the program because I needed to check it with the context of my sentence. But I did not understand many grammar formulas". S1: "I did not change all suggestions given by PWA because I know that my sentence was correct. For example, changing the passive voice into active or in plural and singular correction".
From the data analysis on this topic, it was apparent that two groups of students had different reasons and different ways of using PWA to improve their drafts. The students' views on this matter have a clear cut between the students who had high and low writing achievement.
Finally, it was found that some students might engage superficially with PWA feedback by only clicking the correction. Although students claimed to read and learn from the feedback given, there were conditions that made students engage superficially with PWA such as (1) when they were in a hurry to submit their work, (2) when they confused or did not understand the suggestions, and (3) when they knew they made mistakes instead of errors. The students' representative quotes are as follow: S6: "I often paid attention to the suggestions given by PWA, but when I was in a rush and had no idea about the errors, I preferred to click the correction directly and followed the suggestion". S3: "I mostly read the correction first and learn from it but if I knew the errors were mistakes, I usually just clicked on the correction to change them. Also, when I found that there was a sticky sentence in my paragraphs, I often to ignore it or just change it once accordingly because it did not provide clear suggestions". S1: "Students' engagement with PWA was different for each student. When I was doing my group work on the first assignment, few of my friends directly clicked all of the corrections given by PWA without actually trying to find out whether it is true or not".
The statements suggests that there were different factors that made students engage superficially with the program, in other words the students did not made more effort to deal with the feedback given and made only simple action to improve their drafts. By analyzing the students' responses, the needs of PWA in class were identified. Firstly, the students unanimously appreciated the change in feedback practices that resulted in a more effective time in class. This perception was reinforced by the impression they got from the review session. Because the feedback from PWA and teacher were done outside of the class hour, the teacher can use more time in class for review or practice session that surely benefited the students. It is also worth noting, that the change in feedback was seen as the better way to be done for a big class. Indeed, AWE has been suggested to be used because the program can save time in a big class (Wilson & Czik, 2016;Yinghui & Dan, 2015). Furthermore, having the teacher to explain the PWA's impact on the students' drafts, the students could be careful in editing their errors and learn more about the expected writing performances. Other studies done using different AWE programs also suggested the same trends. An additional teacher feedback or extra time to guide or clarify the result of AWE to students' drafts is used to minimize the students' doubt about the program (Parra & Calero, 2019). Our study highlighted explicitly that the presence of additional review session was indeed needed and valued greatly by the students with different writing achievements. This strategy used by the teacher is also in line with another study that stated if the impact of AWE is influenced by several dynamic factors, one of which is the strategy for using the program (Hockly, 2019). Certainly, after PWA and teacher feedback were given, having an additional review session that discusses the students' errors and expected writing performances can be one strategy to be implemented in class.
Secondly, our findings highlighted the need and usefulness of grammar, vocabulary, punctuation, and spelling feedback to improve the students' drafts. This study resonances with previous study indicating the usefulness of AWE program to help the students reduce the grammatical errors (Huang & Renandya, 2018;Kellogg et al., 2010;Parra & Calero, 2019), help students improve their vocabulary (He, 2016;Huang & Renandya, 2018), and mechanics errors (Huang & Renandya, 2018;Kellogg et al., 2010;S. Wang & Li, 2019) on the students' drafts. Due to the feedback given by AWE, the students also valued positively the teacher feedback on the content and organization aspects of writing. This study support many previous studies on the implementation of teacher feedback with AWE program (Crusan, 2015;Wang, 2015;Wilson & Czik, 2016). Especially for teachers to focus more on the area undetected by PWA, not only on the linguistic but also content or organization aspects.
However, unlike the high-achieving students who valued grammar feedback, students with low writing achievement valued spelling or punctuation feedback greatly as well. This finding can be linked to the students' achievement or proficiency in English. From Table 1, we can also see that the three students with low writing achievement had lower scores in mechanics, vocabulary, and grammar compared to S1, S2, and S3. Indeed, the impact of English proficiency on the type and amount of errors produced by EFL students indicates that students with low English proficiency appear to make more punctuation, spelling, or capitalization errors than those with higher English proficiency (Fati, 2013). Thus, this result complements many previous studies on the potential of AWE to be used by lower level students (Fati, 2013;Liao, 2016;Wang et al., 2013).
Despite the students' positive perceptions of needing the program, the error corrections might only enhance the students' drafts when the students used PWA, and it might not necessarily enhance the students' performances for a long-term effect. The students' achievement test in Table 1 shows some of the students' low score despite using PWA for their writing assignments. Thus, the students may need more opportunities with PWA and teacher feedback. The longer use of the program might yield a different finding. In regards to the students' performances and perceptions, this study complements previous studies that have found a discrepancy between students' positive views and their performances (Huang & Renandya, 2018;Kellogg et al., 2010). It is highlighted that feedback usage is different from feedback provision (Kellogg et al., 2010). Although feedback is given, there is a chance that the students did not actually use it, thus, produce the same errors on different assignments (Zhang, 2016).
Thirdly, the students also appreciate PWA feedback as it could build the students' confidence. Related to this topic, the students' achievement might not be the indicator of whether different students perceive PWA as a self-confidence builder or not because almost all students made this claim. One reason that might be taken into a count is the teacher feedback. Teacher feedback was given only once, hence the students may feel more feedback or proofreaders. Thus, they felt at ease to know this program's potential. Students who used AWE before teacher assessment also mentioned that such a program is a self-confidence builder (Chen & Cheng, 2008). Similarly, machine feedback has been found to increase students' self-confidence because the students could be sure that his/ her composition would not be too bad for the teacher to judge (Wang, 2015). Another consideration is the feedback features of PWA that helped the students to reduce their errors. Indeed, different AWE programs can have different  Jurnal Ilmu Pendidikan (JIP) 49 e- ISSN: 2442-8655 Vol. 27, Issue 1, June 2021 impact to the students. An AWE program that fails to explain students' errors will only make students give negative views over the program (Cushing Weigle, 2011). When an AWE program made unclear or confusing feedback explanations, students will be also unlikely to favor the program (Zhang, 2016).
G. Students' Engagement with PWA to Improve Their Drafts From data analysis, it is found that students utilize PWA differently to improve their drafts for different reasons. The high-achieving students used PWA mostly to reduce errors for their drafts or assure them about their drafts before submitting them to the teacher. Meanwhile, the low-achieving students used PWA not only because they needed it to reduce their errors but also because they needed more feedback. As a result, they were found to use PWA for their first drafts and third drafts. The difference in actions were reinforced by the students' English proficiency and teacher feedback which was only given once.
Indeed, it was found from the data analysis and students' achievement result (see Table 1) that the low-achieving students produced more errors than another group of students in this study. Not to mention that these students were not English-major. It is no surprise if students with low writing proficiency tend to make more errors on their writings (Fati, 2013). As a result, students with low proficiency were more likely to use or trust AWE more due to their inadequate language knowledge (Huang & Renandya, 2018). Yet, they were also vulnerable with PWA feedback as there were undetected errors found in PWA. Another study finds that low-achieving students can make new grammar errors when they attempt to revise their content, meanwhile, AWE might not help the students to detect all their errors (Huang & Renandya, 2018). The students' positive perception on the teachers' direct corrective feedback on the students' linguistic errors can also indicate the students' knowledge of grammar to this matter. Similarly, it is found that direct corrective feedback from the teacher can affect the students' performances positively (Budianto et al., 2017). However, because the teacher only gave feedback on the students' second drafts, students with low writing achievement felt the need of having more feedback for their third drafts, thus they used PWA. The students with low writing achievement seemed to be benefited more with teacher feedback because PWA could not help them fix their poor sentence constructions. Although the implementation of AWE for beginner-level students is suggested Chen & Cheng, (2008), PWA was unable to help the students with poor sentence construction. PWA did not give feedback or correct the errors on this matter Similarly, although, this result complements many previous studies on the potential of AWE to be used by lower-level students (Fati, 2013;Liao, 2016;P. Wang, 2015), it is important to be aware that these students might also be benefited more from teacher feedback. The low-level students in another study also yield the same need in terms of teacher feedback (Yinghui & Dan, 2015). This study also supports many previous studies on the claim that students' engagement with the AWE program is directly affected by the teacher's strategy in class (Y. J. Wang et al., 2013;Warschauer & Grimes, 2008).
In line with the previous discussion on the grammar feedback, the high-achieving students might benefit more than the low-achieving students in using PWA. They could reduce and be critical with their linguistics errors when using PWA as well as reducing the undetected errors with teacher feedback. Also, improve the content from teacher feedback. Similarly, previous findings also found that student with higher level proficiency was better in revision process because they can focus on language and content of their writings (Zhang, 2020) and think of how to make effective revisions based on AWE feedback (Zhang & Hyland, 2018). The findings on the highachieving students in this study also supports how the implementation of teacher and AWE program is suggested (Cotos, 2010;Wang, 2015;Wilson & Czik, 2016).
Finally, when the students were asked about their experiences with PWA, most students mentioned that they corrected their errors and learned from the explanation given by PWA. In a certain situation however, the students were found to engage superficially with PWA. This study supports many findings that also find students who engage superficially or perform a low level behavioral engagement with AWE program (Chen & Cheng, 2008;Zhang, 2016;Zhang & Hyland, 2018). Many students have been found to overlook AWE feedback when explaining the errors makes the students feel confused (Zhang, 2016). Different students also engaged differently with AWE feedback due to some factors such as the students' English level or proficiency, the strategy of learning, and the students' beliefs about the given feedback (Zhang & Hyland, 2018). Indeed, there are complex factors that influenced the students' responses toward implementing a certain AWE program (Warschauer & Grimes, 2008). The result of this study also supports previous researchers that discover the undetected errors in AWE programs (Crusan, 2015;Nova & Lukmana, 2018) or poor explanation given by the program to a certain feedback (Zhang, 2016). Like many computer programs, there is always a defect found in the system (Zupanc & Bosnić, 2015). Thus, this finding supports countless studies suggesting using the AWE program with teacher feedback due to its drawbacks (Hockly, 2019;Parra & Calero, 2019;Wilson & Czik, 2016). Conclusion Students with different writing achievements have a positive view of PWA to improve their drafts. They perceive it positively because with PWA, teachers can use class time more effectively. PWA also has useful feedback (grammar, mechanics, and vocabulary choice feedback) that helps improve students' concepts. Furthermore, revising with PWA gives students confidence in their draft. Then, students with high writing achievement used PWA for the first draft because there were not many corrections that affected the content. In contrast, students with low writing achievement chose to use PWA for the first and third drafts because they were still revising the content based on teacher input. They also need more teacher feedback to correct their poor sentence construction because PWA cannot solve this problem. Finally, although most students claim to learn from PWA feedback, it is possible that students only superficially engage with the program by clicking on corrections directly without trying to understand or see suggestions. However, this study involved a small number of students and only investigated student perceptions, so it may not be appropriate to generalize to a larger context. Future research may wish to include more students with different writing achievements using more advanced or up-todate AWE programs.