Many visual reinforcement learning (RL) algorithms learn representations by matching latent distances to a behavioral distance induced by reward and transition similarity. The choice of the latent distance can strongly affect performance: using fixed, pre-specified global norms (e.g., $\text{l}_p$ norms or other hand-designed metrics) may be overly restrictive to capture the behavioral distance. In contrast, unconstrained pairwise distances may admit degenerate solutions that drive the metric loss down without improving the representation. To address this gap, we introduce PAMD: Pairwise Adaptive Mahalanobis Distance, which parameterizes a positive-definite, pair-conditioned metric for measuring latent state similarity. PAMD is a simple plug-in for existing bisimulation-based methods, offering a more expressive yet structured alternative to fixed, pre-specified latent distances. We empirically validate our method on visual MuJoCo continuous-control tasks, where final performance of several recent bisimulation-based RL algorithms is substantially improved when equipped with the distance we propose.
Blogger's Review: The PAMD method introduces a flexible Mahalanobis distance, addressing the limitations of traditional fixed metrics and providing a new perspective for visual reinforcement learning. This approach not only enhances performance but also maintains model interpretability, making it a valuable avenue for future research and application.