Trend-aware Thompson sampling algorithm for solving multi-armed bandit problems in e-commerce environment

School of Computer Science, Northwestern Polytechnical University, Xi’an, Shaanxi Province, 710129, China

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

Multi-armed bandit algorithms (MAB) have been widely used to make sequential decisions in e-commerce environment such as recommendation systems, online advertising and dynamic pricing strategy adjustments. Among these traditional algorithms, Thompson Sampling (TS) has a superior performance. However, in non-stationary MAB problems, traditional TS algorithm lacks the ability to perceive recent performance changes and differentiate the timeliness of information acquisition. Aiming at compensating these shortcomings, the author proposes an optimized version of traditional TS algorithm: Trend-aware TS algorithm. Through dynamically adjusting variance with 3 mechanisms: trend tracking, ranking adaptation and temporal decay, it avoids premature abandonment of potentially high-quality options which provide stable selection for arms with good recent performance while giving more exploration opportunities to poorly performing arms. In order to verify the effectiveness of this method, the author set up the experiment on the E-Commerce Sales Dataset, comparing the cumulative regret, moving average regret per round and optimal arm selection rate with traditional TS. The results demonstrated that the overall performance of the Trend-aware TS algorithm improves 18.85%, but the improvement declined as the number of rounds increased.