Abstract
Monte Carlo Tree Search (MCTS) is a cornerstone algorithm for online planning, and its root-parallel variant is widely used when wall clock time is limited but best performance is desired. In environments with continuous action spaces, how to best aggregate statistics from different threads is an important yet underexplored question. In this work, we introduce a method that uses Gaussian Process Regression to obtain value estimates for promising actions that were not trialed in the environment.
Method
We perform a systematic evaluation across 6 different domains, demonstrating that our approach outperforms existing aggregation strategies while requiring a modest increase in inference time.
// Example code: Value estimation using Gaussian Process
GaussianProcess gp;
gp.fit(data);
value_estimate = gp.predict(action);Blogger's Review: The proposed Gaussian Process aggregation method offers a novel approach to enhancing the performance of Monte Carlo Tree Search in continuous action spaces, showcasing the potential of statistical aggregation in improving algorithm performance while only requiring a slight increase in inference time. This research lays the groundwork for future decision-making in complex environments.