In traditional deep reinforcement learning, the trained policy is often viewed as a black box. This paper explores how to make it explainable by rewriting it as an executable logic program. We present a three-stage post-hoc transformation: first, we extract a frozen proximal policy optimization teacher, then induce an ordered rule list from its decisions in the manner of classical relational learning, and finally emit the result as a Prolog program, ensuring that every decision can be executed by an off-the-shelf logic engine.
The subsequent expansion stage allows for editing the rule base, accepting changes only when policy evaluation certifies a return increase.
We prove four guarantees: a return-loss bound makes the distilled program a machine-checkable certificate in a finite Markov decision process, and the expansion loop improves monotonically and terminates.
In the continuous-observation setting, we address the feasibility of conversion: the propositional threshold instantiation converts the network to arbitrary fidelity as the resolution B grows, with disagreement O(1/B) and a return gap closing at the same rate. A matching lower bound shows that the cost is exponential in the observation dimension for an oblique decision boundary.
In a two-room key-and-door task with 16,944 reachable states, the expanded Prolog program achieves exact optimal return in every seed and exceeds the stochastic teacher on exact return in ten out of ten seeds under a budget-capped regime.
In three continuous-control tasks, the emitted program substitutes the network, matching the neural teacher within noise on Acrobot with eleven clauses and recovering about 97% of its return on CartPole, while partially recovering on the more finely controlled LunarLander, exactly as the exponential lower bound predicts.
Blogger's Review: This paper successfully transforms the black box nature of deep reinforcement learning into an interpretable rule-based system through logic programming, paving the way for greater transparency and verifiability in reinforcement learning. Such research not only has theoretical significance but also enhances the credibility of decision-making processes in practical applications.