Background and Context
Generative AI (GenAI) tools are increasingly used in programming courses, but we have limited evidence about how brief instruction can foster responsible, learning-oriented use.
Objectives
We evaluate "AI-Lab", a scaffolded GenAI literacy intervention, asking how students' self-reported GenAI usage and their openness and comfort using GenAI for conceptual, debugging, and homework tasks change after participation.
Methods
Across two semesters in three CS courses and one first-year engineering course at a U.S. university, we deployed the "AI-Lab" (pre-lab orientation, in-class critique of GenAI outputs, and a required homework reflection), collecting paired pre/post surveys (Perception N=831; Usage N=826) and six post-intervention focus groups. Primary inferential analyses used the three CS courses (N=778 and 773, respectively). We analyzed survey shifts with paired non-parametric tests and focus groups via thematic analysis.
Findings
Openness increased for conceptual questions and homework help, and comfort increased for conceptual, debugging, and homework scenarios. Self-reported frequency of GenAI use for homework and projects remained stable, while self-reported use for debugging increased. Focus group participants described adopting more iterative prompting strategies, becoming more skeptical of correctness, and articulating clearer boundaries around integrity and dependence.
Implications
A short, structured intervention can shift students' reported comfort with and willingness to use GenAI and influence the strategies they describe for engaging with it without increasing overall self-reported use on graded work. These results motivate future work triangulating surveys with behavioral traces and learning measures.
Blogger's Review: This study highlights how structured interventions can effectively enhance students' capabilities and critical thinking in using Generative AI. Future research could further explore how to integrate quantitative and qualitative data to provide comprehensive guidance for course design.