Scientific research increasingly relies on large, heterogeneous data sources, driving interest in retrieval‑augmented generation (RAG) systems that offer natural‑language access to scientific knowledge and support research workflows. Researchers are testing these systems as natural‑language interfaces for document search and for generating analysis code or pipeline components. At the same time, concerns about data privacy and control over research infrastructure have spurred interest in open‑weight models and open‑source deployments hosted within research institutions.\ In astronomy, this development follows a long history of computational infrastructure, from archival databases and SQL‑based systems to LLM‑assisted research tools. This paper presents a domain‑expert evaluation of AquiLLM, an offline, open‑weight RAG‑LLM platform designed to help scientific groups use and preserve tacit and formal knowledge. We define faithfulness as the degree to which generated responses stay grounded in retrieved scientific context, avoiding unsupported claims or omissions.\ We conduct an astronomy case study evaluating AquiLLM on both retrieval and scientific analysis tasks. Results show that AquiLLM is most reliable on explicit, retrieval‑oriented questions that are directly grounded in the RAG collection, while faithfulness degrades for queries requiring synthesis or ambiguity resolution. These findings highlight both the promise and the limitations of open‑weight RAG‑LLM systems for scientific research and underscore the importance of domain‑expert evaluation beyond standard benchmark leaderboards.\ Review: The paper offers a thorough, real‑world assessment of open‑weight RAG‑LLM performance, reminding us that convenience must be balanced with rigorous checks on the truthfulness and traceability of generated content.