We evaluate how agentic AI models handle a full data‑science pipeline and how feedback improves their performance. In the experiment a writer model equipped with specific tools downloads, cleans, analyzes a dataset and writes a short paper. A grader model then executes the project, scores it, and the writer revises its submission. This loop runs for three rounds.
Seven models from Anthropic, OpenAI, Google, DeepSeek and Z.AI are tested. Each model acts as both writer and grader, yielding 49 writer‑grader pairs, each run three times. Out of 147 initial submissions, 107 successfully run and render results.
The average score rises from 59 % in the first round to 76 % after the final feedback. By writer, the improvement ranges from 3 percentage points for Anthropic’s Opus 5.5 to 46 points for Anthropic’s Haiku 4.5; by grader, it ranges from 13 points for Google Gemini 3.8 Flash to 32 points for Anthropic’s Haiku 4.5.
When a model grades its own work, the score is about 9 percentage points higher than the average given by other models. Roughly half of the round‑to‑round gain can be attributed to the grader’s own noise. Feedback generally yields the highest marks, but even without feedback, merely allowing revisions or providing extra budget (time or tool calls) leads to substantial gains for some models, though not uniformly.
Overall, the benefit of the iterative loop depends on which model occupies which role; weaker models rarely improve stronger ones.
Review: The study highlights the promise of language models in end‑to‑end data‑science tasks and shows that multi‑round feedback and self‑revision can markedly boost quality, yet effective role assignment remains a critical factor.