Edge‑AI model selection is often driven by a single metric—accuracy, latency, memory, energy consumption, or safety—although a deployable language model must balance all five. This paper addresses a fundamental question: do small language models (SLMs) trained natively offer a more sustainable edge deployment than large language models (LLMs) that are compressed via post‑training quantization?
We introduce a Holistic Sustainability Score (HSS) organized around the triple‑bottom‑line:
- Economic pillar: capability (zero‑shot benchmarks) and system efficiency (latency, throughput, peak VRAM, energy)
- Environmental pillar: operational GPU energy
- Social pillar: robustness against harmful prompts (attack success rate)
The study evaluates five BF16‑based SLMs and five LLMs under different quantization schemes—BF16, INT8, NF4 4‑bit, GPTQ 4‑bit, and GGUF Q4—yielding 30 measured configurations. Capability is measured on five zero‑shot tasks, efficiency on latency, throughput, peak VRAM and energy, and safety is approximated by the attack success rate on five harmful prompts.
Findings:
- The top‑scoring model is Qwen3‑30B‑A3B/GGUF Q4 with an HSS of 93.38, followed by Mistral‑Small‑24B/GGUF Q4 (92.40).
- The highest‑ranked SLM is Phi‑4‑mini/BF16 (89.49).
- Thus, the hypothesis that native SLMs are universally the most sustainable edge choice is not supported; optimized quantized LLMs can win overall, while SLMs remain competitive thanks to lower resource demands.
Conclusion: Quantization is a systems‑level choice rather than a monotonic precision‑efficiency trade‑off, and the HSS is relative to its comparison pool and proxy definitions. Future work should refine metrics and explore cross‑model collaborative optimizations.
Review