In partially observable reinforcement learning (POMDP), recurrent networks are typically trained jointly with the policy or value function to encode temporal information. This work asks whether the recurrent module really needs to be learned or if a fixed recurrent network can supply sufficient memory. Inspired by Reservoir Computing (RC) and Generalized Reservoir Computing (GRC), we use a simple architecture: a frozen recurrent network as a feature extractor followed by a trainable nonlinear read‑out. We evaluate the method on tasks with different memory demands, observation modalities, and action spaces. Surprisingly, the frozen recurrent network can retain information over hundreds of steps and achieves control performance comparable to fully trainable models while using less than half the trainable parameters. These findings suggest that learning the recurrent memory is not always required for solving POMDPs, and frozen recurrent networks serve as a simple, parameter‑efficient, and surprisingly strong baseline. Source code: https://github.com/Aphlios/FRN-POMDP
Review