In modern data infrastructure, the success of coding agents relies not only on model quality but also on the skills and environment files that instruct agents on how to utilize a system. This paper explores how to optimize these artifacts for agents operating within a branching lakehouse, Bauplan. Our study shows that using headless APIs and Git-like data primitives exposes data workflows through code, branches, commits, and merges. We observe that a branching lakehouse transforms data-agent evaluation from an output-matching problem into a state-verification problem: agent-generated pipeline code induces concrete, inspectable lakehouse changes.
We propose a data-centric optimization pipeline that generates task-verifier pairs, executes candidate skills in isolated sandboxes, and scores trajectories using both trace-level signals and programmatic checks over lakehouse state. In preliminary evaluations on hundreds of tasks, optimized skills improved held-out rewards by up to 28.6%. These results suggest that write-path data workflows provide a useful substrate for optimizing agent skills beyond read-only tasks.
Blogger's Review: This paper offers an innovative perspective by shifting the evaluation of data agents from traditional output matching to state verification for more efficient skill optimization. This approach not only enhances agent performance but also provides new insights for future designs of data-driven systems. The isolation sandbox execution and multidimensional scoring mechanism merit further exploration in broader applications.