Optimizing Ad Recommendations Using A Bayesian Multi-Armed Bandit Approach

Department of Mathematics, University College London, London, WC1E 6BT, United Kingdom

eugene.zeng.23@ucl.ac.uk

Abstract

The contextual multi-armed bandit problem in online advertising recommendation represents a core challenge in recommender systems. Traditional approaches, including Linear Upper Confidence Bound (LinUCB) and Feature-based Collaborative Filtering (FeatureCF), demonstrate limitations in exploration-exploitation balance and high-dimensional feature modeling: Upper Confidence Bound (UCB) suffers from excessive exploration due to its upper confidence bound strategy, while FeatureCF struggles with convergence due to inefficient gradient updates. To address these deficiencies, this paper proposes the Bayesian Multi-Armed Bandit (MAB), which enhances performance through a three-layered progressive strategy: employing Thompson Sampling for precise reward estimation, integrating a temporal decay factor to dynamically adjust exploration bias, and utilizing a cold-start strategy to accelerate initial learning. Simulation experiments on the Criteo dataset over 10,000 steps reveal that Bayesian MAB outperforms its counterparts, achieving a final Click-Through Rate (CTR) of 0.4407, an optimal arm selection rate of 0.3682, and a Gini coefficient of 0.5731, demonstrating a superior exploration-exploitation balance. The effectiveness of Bayesian MAB in overcoming the shortcomings of traditional methods offers an efficient solution for online advertising recommendation.