Multi-Armed Bandit Based Traffic Signal Control for Congestion Management

School of Mathematical Sciences, University of Science and Technology of China, Hefei, 230026, China

baike@mail.ustc.edu.cn

Abstract

Against the background of rapid urbanization and continuous growth of vehicle numbers, traffic congestion has become increasingly prominent, causing serious negative impacts on urban life and economic development. Traditional traffic signal control methods struggle to adapt to dynamic and complex traffic flows. Although reinforcement learning has brought new opportunities to this field, it faces challenges such as high computational costs, large data requirements, and slow convergence rates. This paper focuses on the Multi-Armed Bandit (MAB) model algorithm, using the Simulation of Urban Mobility (SUMO) traffic simulation software to build a lane model highly similar to reality. Different signal timing plans are set up as "arms" in the MAB model, with vehicle waiting time as "reward". The performance of algorithms such as Explore Then Commit (ETC), Upper Confidence Bound (UCB), Asymptotically Optimal Upper Confidence Bound (asUCB), and Thompson Sampling (TS) in intelligent traffic signal control scenarios is compared. The study finds that the TS algorithm performs best in reducing cumulative regret and vehicle waiting time, providing an effective reference for optimizing actual traffic signal control strategies. The characteristics of other algorithms also provide directions for subsequent algorithm improvements, contributing to the development of intelligent traffic signal control technology.