Explainable Reinforcement Learning (XRL) aims to enhance the transparency and interpretability of Reinforcement Learning (RL) policies, which is crucial in safety-critical and human-centric scenarios. However, much of the current research relies on user studies, focusing on specific audience needs and lacking shared evaluation metrics. In contrast, logic-based approaches within eXplainable Artificial Intelligence (XAI) offer compact and human-readable abstractions of decision-making, yet systematically quantifying the explainability of logical representations remains an open issue.
This work seeks to advance the state of the art in XRL by introducing objective and planning-oriented metrics for policy explainability in RL contexts. Simultaneously, it contributes to logic in XAI by providing a principled method to quantify the explainability of logical rules, moving beyond common-sense evaluations and simple propositional fragments. We utilize Inductive Logic Programming (ILP) to extract symbolic representations of RL policies and define a novel set of explainability metrics, such as activation rate, feature coverage, syntactic distance, and semantic distance. These metrics quantify the alignment between symbolic rules and agent behavior, the role of features in decision-making, and the evolution of policies during training across single-agent and multi-agent RL settings.
Experiments conducted in various RL domains demonstrate that the proposed metrics illuminate action-specific learning dynamics beyond global return, offer fine-grained insights into domain features compared to classical global feature importance estimation methods, and reveal coordination, specialization, and adaptation patterns in Multi-Agent Reinforcement Learning (MARL). Moreover, they provide crucial insights for the transfer and generalization of action-specific policies.
Blogger's Review: This research significantly enhances the understanding of reinforcement learning agent behaviors by introducing new evaluation metrics, especially in multi-agent settings. It lays a foundation for the further development of explainable reinforcement learning, warranting attention to its potential and challenges in practical applications.