Reinforcement Learning (RL) policies can be unsafe and prone to attacks, making reliability a significant challenge. Current automated testing methods focus only on selected environments, scenarios, and RL algorithms. To tackle this issue, we propose a comprehensive framework for testing both single- and multi-agent RL policies under varying conditions. Our implementation, Gimitest, is an open-source tool that supports various gym frameworks and allows modifications of their integrated components. This article details the framework and Gimitest's functionality and architecture, showcasing its effectiveness in testing multiple RL policies in environments like the official Farama Gymnasium and PettingZoo.
// Example code: Basic usage of Gimitest
#include <gimitest/gimitest.h>
int main() {
Gimitest tester;
tester.initialize();
tester.run_tests();
return 0;
}Blogger's Review: Gimitest offers an innovative solution for testing RL policies, especially in multi-agent setups, demonstrating its broad applicability in practical testing. This empowers researchers and developers with a more efficient tool to ensure the safety and stability of RL strategies.