Leveraging AI-Powered Writing Assistants to Enhance L2 Writing Proficiency: A Mixed-Methods Study
Keywords:
AI-powered writing assistants, Grammarly, L2 writing proficiency, Automated feedback in CALLAbstract
This mixed-methods study investigates the efficacy of Grammarly, an AI-powered writing assistant, in enhancing L2 English writing proficiency among 60 intermediate-to-advanced learners at Ilam University. Over a 12-week intervention, an experimental group (n=30) used Grammarly for essay drafting and revision, while a control group (n=30) received traditional teacher and peer feedback. Quantitative analysis of pre/post writing assessments revealed significantly greater improvements in the experimental group for linguistic accuracy, coherence, and syntactic complexity (p<0.001; large effect sizes: d=1.40–1.75), validated via rubric scoring and Coh-Metrix text analysis. Behavioral analytics showed revision frequency (mean=18.3/essay) and time-on-task predicted 54% of writing gains (p<0.001). Qualitative themes from semi-structured interviews (n=15) highlighted enhanced self-efficacy, motivation, and trust in AI feedback for grammar/style, though learners emphasized the need for complementary human guidance on higher-order concerns (e.g., argumentation). Results demonstrate Grammarly’s effectiveness in promoting writing proficiency and engagement, advocating for an integrated AI-human feedback approach in L2 pedagogy.
Downloads
References
Abbott, M., & Abbott, W. (2024). Artificial intelligence or nursing student? revisiting clues in the connectives. Nurse Educator, 49(6), 306-309. https://doi.org/10.1097/nne.0000000000001696
Abduljawad, S. A. (2024). Investigating the impact of ChatGPT as an AI tool on ESL writing: Prospects and challenges in Saudi Arabian higher education. International Journal of Computer-Assisted Language Learning and Teaching, 14(1), 1-19. https://doi.org/10.4018/IJCALLT.367276
Aghaziarati, A., Nejatifar, S., & Abedi, A. (2023). Artificial intelligence in education: Investigating teacher attitudes. AI and Tech in Behavioral and Social Sciences, 1(1), 35-42. https://doi.org/10.61838/kman.aitech.1.1.6
Alharbi, W. (2023). Ai in the foreign language classroom: a pedagogical overview of automated writing assistance tools. Education Research International, 2023, 1-15. https://doi.org/10.1155/2023/4253331
Aliakbari, M., Barzan, P., & Sayyadi, M. (2025). Automated feedback vs. human feedback: A study on AI-driven language assessment. AI and Tech in Behavioral and Social Sciences, 3(2), 113-126. https://doi.org/10.61838/kman.aitech.3.2.9
Cavaleri, M. R., & Dianati, S. (2016). You want me to check your grammar again? The usefulness of an online grammar checker as perceived by students. Journal of Academic Language and Learning, 10(1), 223-236. https://journal.aall.org.au/index.php/jall/article/view/393
Chen, B., Bao, L., Zhang, R., Zhang, J., Liu, F., Wang, S., & Li, M. (2023). A multi-strategy computer-assisted EFL writing learning system with deep learning incorporated and its effects on learning: a writing feedback perspective. Journal of Educational Computing Research, 61(8), 1596-1638. https://doi.org/10.1177/07356331231189294
Cosentino, G., Anton, J., Sharma, K., Gelsomini, M., Giannakos, M., & Abrahamson, D. (2024). Hybrid teaching intelligence: lessons learned from an embodied mathematics learning experience. British Journal of Educational Technology, 56(2), 621-649. https://doi.org/10.1111/bjet.13525
Dewi, U. (2023). Grammarly as automated writing evaluation: its effectiveness from EFL students’ perceptions. Lingua Cultura, 16(2), 155-161. https://doi.org/10.21512/lc.v16i2.8315
Dizon, G., & Gayed, J. (2021). Examining the impact of Grammarly on the quality of mobile l2 writing. The Jalt Call Journal, 17(2), 74-92. https://doi.org/10.29140/jaltcall.v17n2.336
Dizon, G., & Gold, J. (2023). Exploring the effects of Grammarly on EFL students’ foreign language anxiety and learner autonomy. The Jalt Call Journal, 19(3), 299-316. https://doi.org/10.29140/jaltcall.v19n3.1049
Dogomeo, A. (2023). Factors influencing motivation in Korean adults' English language learning. Asia-Pacific Journal of Convergent Research Interchange, 9(12), 619-628. https://doi.org/10.47116/apjcri.2023.12.50
Ebyary, K. (2022). Error flagging tools and online revision behaviour: evidence from native and non-native speakers. Cdelt Occasional Papers in the Development of English Education, 78(1), 393-421. https://doi.org/10.21608/opde.2022.255403
Ekoniak, M., Scanlon, M., & Mohammadi-Aragh, M. (2013). Improving student writing through multiple peer feedback. Proceedings of 2018 IEEE Frontiers in Education Conference (FIE), 626-628. https://doi.org/10.1109/fie.2013.6684901
Er, E., Akçapınar, G., Khalil, M., Noroozi, O., & Banihashem, S. (2024). Assessing student perceptions and use of instructor versus ai‐generated feedback. British Journal of Educational Technology, 56(3), 1074-1091. https://doi.org/10.1111/bjet.13558
Fan, N. (2023). Exploring the effects of automated written corrective feedback on EFL students’ writing quality: a mixed-methods study. Sage Open, 13(2). https://doi.org/10.1177/21582440231181296
Feng, Z. (2023). Meta-cognitive strategies, automated writing evaluation feedback and English writing performance of Chinese EFL college students. International Journal of Education and Humanities, 11(3), 483-486. https://doi.org/10.54097/ijeh.v11i3.15154
Fitria, T. (2021). 'Grammarly' as a teachers' alternative in evaluating non -EFL students writings. Leksema Jurnal Bahasa Dan Sastra, 6(2), 141-152. https://doi.org/10.22515/ljbs.v6i2.3957
Ghufron, M. (2019). Exploring an automated feedback program ‘Grammarly’ and teacher corrective feedback in EFL writing assessment: modern vs. traditional assessment. Proceedings of the 3rd English Language and Literature International Conference, ELLiC. https://doi.org/10.4108/eai.27-4-2019.2285308
Godwin-Jones, R. (2022). Partnering with AI: Intelligent writing assistance and instructed language learning. Language Learning & Technology, 26(2), 5-24.
Gozali, I., Wijaya, A., Lie, A., Cahyono, B., & Suryati, N. (2024). Leveraging the potential of ChatGPT as an automated writing evaluation (awe) tool: students' feedback literacy development and awe tools integration framework. The Jalt Call Journal, 20(1), 1-22. https://doi.org/10.29140/jaltcall.v20n1.1200
Han, F., & Wang, Z. (2021). Comparison of English teacher feedback and automated writing feedback on the quality of English language learners’ essay revision. Proceedings of The Open/Technology in Education Society and Scholarship Association Conference, 1(1), 1-5. https://doi.org/10.18357/otessac.2021.1.1.44
Kang, J., Kang, C., Yoon, J., Ji, H., Li, T., Hyunmi, M., & Han, J. (2023). Dancing on the inside: a qualitative study on online dance learning with teacher-ai cooperation. Education and Information Technologies, 28(9), 12111-12141. https://doi.org/10.1007/s10639-023-11649-0
Khan, M., Nazim, M., & Alzubi, A. (2024). Exploring Arab EFL learners' attitudes: is Grammarly a game-changer in academic writing classes? Educational Administration: Theory and Practice, 30(4), 1365-3171. https://doi.org/10.53555/kuey.v30i4.1612
Koltovskaia, S. (2022). Postsecondary l2 writing teachers’ use and perceptions of Grammarly as a complement to their feedback. Recall, 35(3), 290-304. https://doi.org/10.1017/s0958344022000179
Kong, S., Lee, J., & Tsang, O. (2024). A pedagogical design for self-regulated learning in academic writing using text-based generative artificial intelligence tools: 6-p pedagogy of plan, prompt, preview, produce, peer-review, portfolio-tracking. Research and Practice in Technology Enhanced Learning, 19, 30. https://doi.org/10.58459/rptel.2024.19030
Laudari, S. (2015). Motivation and learning English: a study on Nepalese public high school learners. Journal of Nelta, 19(1-2), 99-116. https://doi.org/10.3126/nelta.v19i1-2.12084
Liu, M., Calvo, R., Pardo, A., & Martin, A. (2015). Measuring and visualizing students’ behavioral engagement in writing activities. Ieee Transactions on Learning Technologies, 8(2), 215-224. https://doi.org/10.1109/tlt.2014.2378786
Luo, Y., & Liu, Y. (2017). Comparison between peer feedback and automated feedback in college English writing: a case study. Open Journal of Modern Linguistics, 7(4), 197-215. https://doi.org/10.4236/ojml.2017.74015
Marghany, M. (2023). Using artificial intelligence-based instruction to develop EFL higher education students’ essay writing skills. Cdelt Occasional Papers in the Development of English Education, 82(1), 219-240. https://doi.org/10.21608/opde.2023.313623
Maulidina, P., & Wibowo, H. (2022). The use of Grammarly tools to enrich student's writing ability. Lingua, 18(2), 179-189. https://doi.org/10.34005/lingua.v18i2.2246
Minh, A. (2024). Leveraging ChatGPT for enhancing English writing skills and critical thinking in university freshmen. Journal of Knowledge Learning and Science Technology, 3(2), 51-62. https://doi.org/10.60087/jklst.vol3.n2.p62
Miranty, D., Widiati, U., Cahyono, B., & Sharif, T. (2021). The effectiveness of using Grammarly in teaching writing among Indonesian undergraduate EFL students. Proceedings of the International Seminar on Language, Education, and Culture (ISoLEC 2021). https://doi.org/10.2991/assehr.k.211212.008
Miranty, D., Widiati, U., Cahyono, B., & Suzila, T. (2023). Automated writing evaluation tools for indonesian undergraduate English as a foreign language students’ writing. International Journal of Evaluation and Research in Education, 12(3), 1705. https://doi.org/10.11591/ijere.v12i3.24958
Nabizadah, P. (2020). The impact of formative feedback strategies on EFL Afghan students’ writing skills. International Journal of Advanced Academic Studies, 2(2), 77-82. https://doi.org/10.33545/27068919.2020.v2.i2b.88
Nova, M. (2018). Utilizing Grammarly in evaluating academic writing: A narrative research on EFL students’ experience. Premise Journal of English Education, 7(1), 80-96. https://doi.org/10.24127/pj.v7i1.1332
O'Neill, R., & Russell, A. (2019). Stop! Grammar time: University students’ perceptions of the automated feedback program Grammarly. Australasian Journal of Educational Technology, 35(1), 42-56. https://doi.org/10.14742/ajet.3795
Prasetya, R., & Raharjo, D. (2023). Enhancing English language writing skills: an evaluation of the efficacy of Grammarly application. Journal of English Language Studies, 8(2), 320. https://doi.org/10.30870/jels.v8i2.19294
Rababah, L., & Talafha, D. (2024). Unlocking writing potential: assessing the impact of Grammarly on Jordanian EFL students' writing proficiency. Theory and Practice in Language Studies, 14(5), 1486-1492. https://doi.org/10.17507/tpls.1405.21
Ranalli, J. (2018). Automated written corrective feedback: how well can students make use of it? Computer Assisted Language Learning, 31(7), 653-674. https://doi.org/10.1080/09588221.2018.1428994
Roa, A., & Halim, S. (2024). The impact of ai-powered software on second language (l2) writing: a systematic literature review. Research and Innovation in Applied Linguistics-Electronic Journal, 2(2), 138-158. https://doi.org/10.31963/rial.v2i2.4801
Sari, E., & Han, T. (2024). The impact of automated writing evaluation on English as a foreign language learners' writing self‐efficacy, self‐regulation, anxiety, and performance. Journal of Computer Assisted Learning, 40(5), 2065-2080. https://doi.org/10.1111/jcal.13004
Schmidt, R. W. (1990). The role of consciousness in second language learning. Applied Linguistics, 11(2), 129-158. https://doi.org/10.1093/applin/11.2.129
Shahriar, A., & Laboni, F. (2023). Potential success in English writing skills using artificial intelligence “Grammarly”. DIU Journal of Humanities and Social Science, 10(1), 57-75. https://doi.org/10.36481/dw4ecj03
Shiyao, W. (2021). Understanding the effects of automated writing corrective feedback on l2 writing accuracy across proficiency levels. Frontiers in Educational Research, 4(11), 94-98. https://doi.org/10.25236/fer.2021.041117
Suryanto, S., Habiburrahim, H., Akmal, S., Zainuddin, Z., Safrul, M., & Hanani, F. (2024). Scrutinizing the impacts of Grammarly application on students’ writing performance and perception. Jurnal Ilmiah Peuradeun, 12(2), 465-490. https://doi.org/10.26811/peuradeun.v12i2.1235
Talebinamvar, M., & Zarrabi, F. (2022). Clustering students’ writing behaviors using keystroke logging: a learning analytic approach in EFL writing. Language Testing in Asia, 12(1). https://doi.org/10.1186/s40468-021-00150-5
Thi, N., & Nikolov, M. (2021). How teacher and Grammarly feedback complement one another in Myanmar EFL students’ writing. The Asia-Pacific Education Researcher, 31(6), 767-779. https://doi.org/10.1007/s40299-021-00625-2
Thi, N., & Nikolov, M. (2023). Effects of teacher, automated, and combined feedback on syntactic complexity in EFL students’ writing. Asian-Pacific Journal of Second and Foreign Language Education, 8(1). https://doi.org/10.1186/s40862-022-00182-1
Tian, L., Liu, Q., & Zhang, X. (2022). Self-regulated writing strategy use when revising upon automated, peer, and teacher feedback in an online English as a foreign language writing course. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.873170
Tran, T. (2024). Ai tools in teaching and learning English academic writing skills. Proceedings of the Asia call International Conference, 4, 170-187. https://doi.org/10.54855/paic.23413
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. Harvard University Press.
Wali, F., & Huijser, H. (2018). write to improve: exploring the impact of an automated feedback tool on Bahraini learners of English. Learning and Teaching in Higher Education Gulf Perspectives, 15(1), 14-34. https://doi.org/10.18538/lthe.v15.n1.293
Wang, C. (2024). Exploring students’ generative ai-assisted writing processes: perceptions and experiences from native and nonnative English speakers. Technology Knowledge and Learning. https://doi.org/10.1007/s10758-024-09744-3
Wang, Y., & Wang, H. (2024). Mediating effects of artificial intelligence on the relationship between academic engagement and mental health among Chinese college students. Frontiers in Psychology, 15, 1477470. https://doi.org/10.3389/fpsyg.2024.1477470
Wang, Z., & Han, F. (2022). The effects of teacher feedback and automated feedback on cognitive and psychological aspects of foreign language writing: a mixed-methods research. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.909802
Wei, P., Wang, X., & Dong, H. (2023). The impact of automated writing evaluation on second language writing skills of Chinese EFL learners: a randomized controlled trial. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1249991
Wu, T., & Yu, Z. (2024). Bibliometrically and systematically analyzing automated writing evaluation for English learning. Forum for Linguistic Studies, 5(3). https://doi.org/10.59400/fls.v5i3.1907
Zuboff, S. (2019). The age of surveillance capitalism: The fight for a human future at the new frontier of power. PublicAffairs.
Downloads
Additional Files
Published
Submitted
Revised
Accepted
Issue
Section
License
Copyright (c) 2024 Reza Khany (Author); Pooria Barzan (Corresponding Author)
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
