Active Inference-Driven Multi-Armed Bandits: Superior Performance through Dynamic Correlation Adjustments

Faculty of Computer Science and Technology, Qilu University of Technology (Shandong academy of sciences), Jinan, China

^* Corresponding author: 202204370113@stu.qlu.edu.cn

Abstract

In recent years, Multi-Armed Bandit (MAB) algorithms have gained substantial attention due to their effectiveness in real-world applications, such as recommendation systems, autonomous systems, and dynamic resource allocation. Traditional MAB algorithms, such as UCB and Thompson Sampling, often lack mechanisms to incorporate correlations between arms, limiting their adaptability and optimality in complex environments. This paper presents a novel MAB framework that integrates Active Inference through a dynamic Adaptive Influence Factor (AIF) mechanism. The AIF mechanism builds correlation matrices to capture inter-arm dependencies and dynamically adjusts exploration strategies through an influence factor, γ, which adapts over time based on pull counts. This adaptive exploration enhances decision-making in sparse and uncertain environments by leveraging correlations. The proposed framework is evaluated on movie recommendation data, with AIF-based algorithms, particularly AIF-TS, significantly outperforming traditional and correlated bandit approaches in settings with high data sparsity. These results demonstrate that dynamically adjusting exploration based on inter-arm relationships substantially improves performance in real-world applications, where data quality and relationships are often variable. The findings suggest that incorporating inter-arm correlations with active inference can lead to more efficient and effective decision-making in adaptive systems, highlighting the potential of AIF-based MAB algorithms in addressing real- world challenges.