Multi-Agent Contextual Bandits for Multi-Objective Recommendation Systems

First College of Liberal Arts and Sciences, University of Florida, Gainesville, FL, 32612, USA

Ziwen.zhu@ufl.edu

Abstract

In this work, the use of multi-agent contextual bandits to optimize multi-objective recommendation systems in resource-constrained settings is examined. A baseline single-agent model with mean-aggregated rewards, a homogeneous multi-agent system with identical Bayesian Linear Thompson Sampling agents, and a heterogeneous multi-agent framework with a unique contextual bandit algorithm assigned to each objective were the three experimental configurations that were built. To resolve conflicts between conflicting aims, diversity limitations, and criticality-based adaptive weighting were implemented. In comparison to homogeneous and single-agent baselines, the results show that heterogeneous agent configurations produce faster convergence, reach steady state faster within the first 20000 steps, larger suggestion diversity, and more effective specialization across reward targets, with a Generalized Gini Index (GGI) score of around 11 indicating diverse weight across reward features. They do, however, also make agent coordination more difficult. Results indicate that, especially in settings with extremely varied reward distributions, heterogeneous multi-agent systems offer a useful compromise between efficiency and adaptability.