Abstract:

Artificial Intelligence (AI) has become increasingly prevalent in academic writing support through Automated Writing Evaluation (AWE) systems. However, the pedagogical value of AI tools for writing development remains underexplored, particularly regarding their impact on cognition, authorial voice, and long-term writing skills. This study adopted Feedback Intervention Theory and revision heuristics to examine the quality and impact of AI-generated feedback in a Writing for Publication course designed for doctoral researchers.

This pre-post intervention study engaged doctoral researchers in a three-phase editing process. In Phase 1, participants revised their academic papers based on peer review, instructor feedback, and course content. In Phase 2, participants engineered prompts to generate AI feedback using ChatGPT-4, targeting academic register, flow, cohesion, readability, and confidence. They requested specific improvement recommendations with concrete suggestions and supporting resources. In Phase 3, participants compared texts edited during both phases and reflected on the quality, utility, and impact of AI-generated versus traditional human feedback. We measured student awareness, authorial voice preservation, mitigation strategies, writing development, satisfaction levels, revision patterns, and skill-specific learning gains.

AI feedback demonstrated comprehensive coverage of grammar, style, structure, argument coherence, and citation quality—comparable to expert editorial review. However, participants reported significant concerns about voice authenticity, noting that AI-edited texts "didn't sound like them." While acknowledging the high technical quality of AI suggestions, researchers expressed a strong preference for maintaining their distinctive academic voice. These findings highlight a critical tension between technical accuracy and authentic scholarly identity.

Our results suggest that AI tools function effectively as editorial assistants but may inadequately support the development of critical thinking and authentic academic voice. The study raises important questions about moving AI beyond surface-level correction toward genuine enhancement of researchers' writing skills and intellectual development. This presentation discusses evidence-based pedagogical strategies for integrating AI feedback into writing courses that promote effective cognitive strategies when engineering AI prompts, ensure authorial voice remains intact throughout the revision process, and develop higher-order thinking and writing skills beyond mechanical correction. The study contributes to an emerging framework for teachers, researchers, and pedagogical planners to scaffold AI tool adoption in ways that genuinely advance writing development while preserving scholarly identity.

Keywords:

Artificial intelligence, academic writing, peer review, writing pedagogy, authorial voice, ChatGPT-4, automated writing evaluation (AWE), doctoral education, feedback intervention theory, writing development.