Reinforcement Learning (RL) is widely used to enhance the performance of autonomous systems, including the Autonomous Internet of Things (AIoT). However, the trial-and-error nature of RL in real-world environments is costly and risky in some scenarios. As a result, most RL research is conducted in simulation, which introduces challenges related to Sim-to-Real transferability. Evaluating the robustness of Sim-to-Real algorithms and their gap is crucial for improving RL performance in real-world applications.
To address these concerns, we developed a real-world platform for studying RL in AIoT. An agent deployed on an edge device plays video games on a separate host computer via a hardware-emulated keyboard, guided by vision input. This platform uses commercially available components costing less than USD 400, along with two computers. The system's objective of maximizing game scores inherently mitigates safety risks associated with real-world RL deployments.
Experimental results show that the simulation-trained agent suffers a 1160% performance degradation compared to human-level performance after real-world deployment, indicating a significant Sim-to-Real gap. Direct real-world training using the Deep Q-Network (DQN) algorithm achieves approximately 38% of human-level performance after 10 million training steps, demonstrating the feasibility of RL under real-world conditions. These results suggest that the proposed Sim-to-Real benchmark platform provides a substantial foundation for qualitative and quantitative evaluations of RL in real-world AIoT systems.
Blogger's Review: This benchmark platform offers an essential experimental foundation for RL research in the AIoT domain. By reducing costs and safety risks, researchers can evaluate the practical performance of RL algorithms across broader application scenarios, advancing the technology's practical implementation. It will be exciting to see how this platform impacts future RL research and applications.