Humans transfer behavioural knowledge to new tasks instead of relearning from scratch. A reinforcement learning (RL) agent can do the same: known behaviour patterns need only be applied, not relearned. Neuro‑symbolic RL bridges prior knowledge and RL by injecting symbolic knowledge alongside a learned policy. The integration point is crucial; a poor choice can cause hallucinated preconditions, leading to safety and reliability issues in changing environments. We formalise the behavioural knowledge as a precondition Bayesian network (BN) over the agent's structural actions—actions whose legality depends on preconditions, such as picking up a key, grasping a block, toggling a door, or dropping an object. The BN restricts when these actions may fire and is injected into the RL loop in three placements:
- Symbolic verifier: consulted only at inference, firing a structural action only when its preconditions hold.
- Symbolic enforcer: active during both training and inference, governing structural‑action use throughout learning.
- Symbolic learner: folds the knowledge into the network, letting the model learn the restriction and use of structural actions itself.
We evaluate the three variants on two opposite benchmarks: one built on long, ordered planning chains (MiniGrid) and another on continuous manipulation (Fetch). Metrics include solution quality, sample efficiency, and traceability. Results show substantial gains for all placements. On MiniGrid, the symbolic enforcer achieves $98.2\%$ solution quality versus $88.8\%$ for the PPO+RND baseline; the verifier and learner reach $96.5\%$ and $95.3\%$ respectively. In the Fetch domain, the verifier and learner improve sample efficiency by roughly $30\%$ and $35\%$ while preserving high trajectory interpretability. Overall, appropriately embedding a precondition BN into RL markedly enhances agent safety and efficiency without extra supervision.
Review