Requirements engineering underpins systems engineering, and defects in requirements can propagate downstream, harming development outcomes. Large language models (LLMs) show promise for assessing requirement quality, yet prior work indicates that off‑the‑shelf models miss many expert‑identified defects, perform unevenly across quality criteria, and exhibit high run‑to‑run variability.
This study investigates whether injecting expert‑derived systems engineering knowledge can improve LLM‑based requirement quality evaluation beyond raw capabilities. We introduce MARQ, a modular, LLM‑agnostic prototype that layers a criterion‑specific knowledge stack—operationalizing INCOSE quality criteria—on top of any LLM.
MARQ was evaluated on an expert‑labeled NASA requirement set using Claude Opus 5, GPT‑5.6 Sol, and the open‑weight Gemma 3 12B, with 100 executions per model. Median F1 scores rose to 0.80 for Claude Opus 5, 0.62 for GPT‑5.6 Sol, and 0.53 for Gemma 3 12B, compared with baseline scores of 0.55, 0.52, and 0.18 respectively.
The nature of the gains is model‑dependent: Claude Opus 5 improves both defect detection and false‑alarm control; GPT‑5.6 Sol gains detection and execution‑level consistency at the cost of more false alarms; Gemma 3 12B reaches the performance of proprietary models but remains highly variable.
Criterion‑level analysis shows improvements concentrated in Correctness and Conformance, trade‑offs between misses and false alarms within individual criteria, and model‑specific responses across several criteria.
These findings position knowledge infusion as a promising, model‑agnostic route to more trustworthy LLM‑based RE support, while highlighting the need for criterion‑specific interventions and human‑in‑the‑loop review.
Review