While large language models (LLMs) have set benchmarks for zero-shot reasoning, their deployment is cost-prohibitive and environmentally taxing. Small Language Models (SLMs) present a sustainable alternative but are error-prone in tasks requiring complex, multi-hop logical grounding.
We investigate a neuro-symbolic agentic framework to enhance the reasoning capabilities of SLMs, specifically Gemma 3 (1B, 4B) and Llama 3.2 (3B), using the CLUTRR kinship benchmark. Our approach transforms the SLM into a minimalist agent utilizing two specialized tool calls: extract_facts for symbolic triplet extraction and get_hint for expert reasoning via a Relational Graph Convolutional Network (RGCN).
We evaluate these models across two configurations: an Oracle scenario with ground-truth triplets and a Realistic scenario relying on self-extracted knowledge. Our results reveal that while RGCN-derived hints provide a 1.5–2x performance gain over story-only baselines, the system is constrained by the extraction bottleneck and sequential deductive fragility, where early extraction errors compound over multi-hop chains.
Additionally, we identify a "distraction effect" in specific architectures where noisy, self-generated facts degrade performance despite the presence of expert hints. This work characterizes the challenges of symbolic grounding in low-resource agentic systems and provides a roadmap for iterative verification in neuro-symbolic agentic pipelines.
Blogger's Review: This study showcases the potential of small language models in reasoning tasks, particularly through the integration of knowledge graphs to enhance performance. Despite the challenges of extraction bottlenecks and distraction effects, the framework's design offers significant insights for future research, making it a topic worth following.