Most questions reaching an HPC help desk have already been answered in the center's user guides or in tickets filed previously, yet users still open new tickets because the guides are hard to navigate and ticket histories are not searchable by individuals. We present an assistant that draws from both sources: an orchestrator invokes two tools via the Model Context Protocol—a retrieval‑augmented generation (RAG) over live documentation and a Llama‑3‑8B model fine‑tuned with QLoRA on 11,244 query‑resolution pairs mined from historical tickets. Both models run on‑premises or on the university's hosted LLM service to keep ticket data within the institution.
Evaluation on 400 held‑out tickets from Purdue's Rosen Center for Advanced Computing, disjoint from training data and free of near‑duplicates, was performed by two judges independent of the model families that generated the answers. Fine‑tuning improves BERTScore F1 from 0.06 to 0.27 over the base model and aligns best with RCAC's official resolutions. The end‑to‑end system does not surpass its individual tools, although they are complementary: an oracle that picks the better answer per question would achieve a correctness score of 3.41 versus 3.20 for the merged system, and the merge follows the documentation tool in 78% of cases. Returning only the fine‑tuned tool's answer recovers its quality (3.19 vs. 2.66), while a learned selector or LLM verifier fails to beat that tool alone.
Review: This study demonstrates the practicality of combining documentation retrieval with fine‑tuned historical ticket models, especially under strict data‑privacy constraints common in academic settings. Future work could investigate more sophisticated answer‑fusion mechanisms to further boost overall accuracy.