Optimizing Data Filtering in Multi-Armed Bandit Algorithms for Reinforcement Learning

Math Department, Stony Brook University, NY11790, New York, United States

^* Corresponding author: Shengshi.zhang@stonybrook.edu

Abstract

This study investigates the performance of data filtering algorithms in multi-armed bandit (MAB) problems for reinforcement learning applications. It focuses on five algorithms: Epsilon-Greedy (ε- Greedy), Upper Confidence Bound (UCB), Linear Upper Confidence Bound (LinUCB), Thompson Sampling, and Linear Thompson Sampling (LinTS). The algorithms were evaluated in static and dynamic environments using the MovieLens dataset and transferred to binary rewards to measure performance. Each algorithm was tested in simulations with 1,000 interactions, and compared with cumulative reward, accuracy, and adaptability. The experiments involved multiple arms, each with a unique reward distribution, and simulated static (fixed rewards) and dynamic (periodically changing rewards) environments. Result shows that LinUCB and LinTS achieved the highest cumulative rewards in static settings. In dynamic environments, Thompson Sampling demonstrated superior adaptability by adjusting quickly to changing reward structures. Overall, LinUCB and LinTS maintained an accuracy rate of over 36%. This research provides critical insights into the trade-offs between exploration and exploitation, offering theoretical and experimental support for optimizing MAB algorithms in real-world applications such as recommendation systems.