Abstract:

The rapid proliferation of large language models (LLMs) has intensified concerns surrounding academic integrity. While numerous AI detection tools have emerged to identify AI-generated text, their robustness remains an open question. This paper investigates the vulnerability of a widely-used AI detection system, ZeroGPT, by demonstrating how an LLM can learn to evade detection through an iterative, feedback-driven generation process — without being explicitly programmed with evasion strategies.

Unlike prior work that relies on fixed prompt templates or prompt skeletons to generate AI essays in a single pass, we propose an iterative adversarial framework in which an LLM progressively refines its writing style based on feedback from the detector itself. Concretely, we begin by generating AI-written essays using an LLM prompted to respond to TOEFL writing tasks drawn from the TOEFL 11 corpus — a well-established dataset of essays written by non-native English speakers across 11 countries. These AI-generated essays, along with authentic TOEFL 11 essays, are then submitted to ZeroGPT. The resulting detection reports — which indicate which essays were flagged as AI-generated and which human-authored essays were classified as human-written — are fed back to the LLM as a contextual learning signal. The LLM uses this information to identify what linguistic and stylistic patterns trigger detection, and adjusts subsequent essay generation accordingly. This loop is repeated across multiple rounds until ZeroGPT's classification accuracy approaches chance level (approximately 50%), at which point the generated essays are considered effectively indistinguishable from human writing by the detector.

Importantly, this work is not an attempt to facilitate cheating on the TOEFL examination. Rather, we use the TOEFL 11 corpus as a controlled, well-understood benchmark of authentic L2 English writing to rigorously evaluate the fragility of current AI detection tools. Our goal is to expose a meaningful, underexplored vulnerability: AI detectors can be systematically misled when an LLM has access to its own detection feedback.

To further validate our findings beyond automated detection, we conduct a human evaluation in which experienced English instructors are asked to classify a mixed set of authentic TOEFL 11 essays and our iteratively generated AI essays, providing an additional measure of how convincingly human-like our generated essays appear.

Beyond the experimental findings, a central contribution of this work is the public release of our iteratively generated essay dataset as a companion to the TOEFL 11 corpus. This dataset is designed to serve the research community as a challenging benchmark for developing and stress-testing future AI detection models — one that reflects a more realistic threat model than datasets generated by naive one-shot methods.

Our results suggest that current AI detection tools are more fragile than commonly assumed, and that the field urgently needs more adversarially-aware datasets and detection methods to keep pace with the evolving capabilities of generative AI.

Keywords:

AI-generated text detection, adversarial essay generation, Large language models.