The critical role of division of labor (DOL) in enhancing cooperation is well-recognized in real-world applications. Consequently, many cooperative multi-agent reinforcement learning (MARL) methods have incorporated DOL mechanisms to improve cooperation among agents. However, the lack of benchmark tasks specifically designed to evaluate and promote DOL and cooperation has limited the effective development and deployment of such mechanisms.
To address this gap, we propose the Composite Tasks Challenge (CTC), a suite of tasks explicitly designed to require both DOL and cooperation for successful task completion. The CTC tasks are constructed based on two core design principles: 1) DOL is a necessary condition for task success; 2) Failure in any atomic subtask results in failure of the overall task. The first principle emphasizes the necessity of DOL, while the second enforces the importance of cooperation, making both components essential for success in CTC tasks.
We evaluate nine representative cooperative MARL methods on the proposed CTC tasks. Experimental results show that all methods consistently achieve zero test winning rates across all CTC tasks, highlighting the challenge of CTC tasks and the limitations of current methods. To facilitate future research, we also introduce a guiding solution that achieves non-zero test winning rates on all tasks, demonstrating the solvability of the CTC tasks. However, the performance of this guiding solution remains suboptimal, further underscoring the value of CTC tasks as a challenging and meaningful testbed for advancing cooperative MARL research.
Blogger's Review: The introduction of CTC provides a new testing ground for cooperative multi-agent reinforcement learning, emphasizing the importance of division of labor and cooperation in task success. This not only offers researchers an opportunity to evaluate existing algorithms but also points the way for future algorithm improvements.