Inducing cooperation among distributed agents remains a challenging issue in multi-agent reinforcement learning (MARL), especially in social dilemma situations. Here, individual interests are misaligned with the common good, leading to suboptimal group outcomes due to individual rationality. In contrast, humans often achieve cooperation in such scenarios. A common explanation for this cooperative behavior is that individuals possess social preferences. To foster cooperation in MARL, we propose a novel utility function that integrates altruistic preferences (incentives for others' rewards) and fairness preferences (incentives for equality) from social psychology and behavioral economics, termed Altruistic and Fairness Preference (AFP). This reward-sharing mechanism converts both one's own and others' rewards into incentives for cooperative behavior.
We conducted comparative experiments against standard reinforcement learning and inequity aversion agents in two challenging sequential social dilemma games. The results indicate that AFP agents successfully achieved mutual cooperation, yielding greater collective rewards and higher equity than the baselines. To further investigate the progression of AFP during training, we explored the effects of altruistic and fairness preferences on agent behavior. The findings suggest that altruistic preferences encourage agents to contribute to public goods, while fairness preferences induce mutual behavior among agents.
Blogger's Review: This study introduces the AFP mechanism, offering a novel approach to cooperation in multi-agent systems. By incorporating altruistic and fairness preferences, it significantly enhances cooperative efficiency, which warrants further exploration in practical applications. The experimental results underscore the importance of social preferences in reinforcement learning, suggesting potential for application in more complex environments.