Optimizing Neural Spike Train Prediction Using Contextual Bandit Algorithms Within Dqn Frameworks

School of Computer Science, China University of Geosciences, Wuhan, 430070, China

q3037433325@cug.edu.cn

Abstract

This study addresses key challenges in applying Deep Q-Networks (DQN) to neural spike train prediction by introducing a novel Contextual Bandit-enhanced DQN (CB-DQN) approach. This research focuses on three critical limitations of standard DQN: inefficient exploration, slow convergence, and poor utilization of sparse reward signals. This research method integrates contextual bandit algorithms to identify neural activity patterns, implement context-specific exploration, and combine DQN with bandit value estimates. Using simulated neural data generated with Brian2, this research demonstrates that CB-DQN achieves 5-7% improvements in accuracy, precision, recall, and F1 score compared to standard DQN. While CB-DQN requires more computational resources during training, it delivers 20% faster prediction times and more stable Q-value estimates during inference. Particularly effective at utilizing sparse rewards, CB-DQN shows accelerated learning curves and better performance on rare spike events. These results demonstrate the potential of context-aware reinforcement learning in computational neuroscience, offering a promising framework for modeling complex neural dynamics with improved biological plausibility.