Multi-Agent Reinforcement Learning in Starcraft: Algorithmic Advances and Collaborative Intelligence Challenges

Leicester Internatioanl Institute, Dalian University of Technology, Dalian, China

blah330903@mail.dlut.edu.cn

Abstract

Multi-Agent Reinforcement Learning (MARL) shows promise in complex dynamic tasks but faces challenges in non-stationarity, credit assignment, communication efficiency, and scalability. This paper evaluates MARL algorithms on the StarCraft Multi-Agent Challenge(SMAC), analyzing their performance in heterogeneous cooperation, dynamic policy adaptation, and large-scale scenarios. Two mainstream approaches are compared: Q-value Mixing(QMIX), which couples global and local decisions via a nonlinear mixing network, achieves a 96.9% win rate in symmetric scenarios but drops to 39.1% in non-monotonic tasks due to its monotonicity constraint. Multi-Agent Proximal Policy Optimization(MAPPO), leveraging centralized critics and parameter sharing, outperforms QMIX in large-scale scenarios, demonstrating the adaptability of policy gradient methods in high-dimensional action spaces. Experiments show MAPPO achieves zero-standard-deviation wins in 81.25% of test scenarios, while QMIX suffers from high variance (12.5) due to insufficient policy coupling. Future work should overcome monotonicity constraints and global state dependence, exploring adaptive cooperation frameworks and hierarchical architectures to enhance scalability and cross-task transferability. Integrating causal reasoning and distributed training could enable more efficient multi-agent coordination in unmanned swarms and smart city applications.