Towards Efficient Deep Reinforcement Learning: A Comprehensive Review of Optimization Techniques

School of Computer and Electronic Information /School of Artificial Intelligence, Nanjing Normal University, Nanjing, Jiangsu, China

19240722@njnu.edu.cn

Abstract

Deep reinforcement learning (DRL) has achieved remarkable success in various domains, including robotics, game playing, and autonomous systems. However, optimizing DRL models remains a significant challenge due to issues such as sample inefficiency, instability, and high computational costs. To address these challenges, researchers have explored various optimization strategies that enhance learning efficiency and model performance. This paper provides a comprehensive review of optimization methods in DRL, focusing on hyperparameter optimization, structural optimization, and algorithm optimization. It explores common techniques for hyperparameter optimization, including Bayesian optimization, grid search, gradient optimization, and evolutionary algorithms, discussing their strengths, weaknesses, and suitable application scenarios. The paper also examines structural optimization, highlighting key mechanisms such as attention mechanisms and generative adversarial networks (GANs), and their impact on improving model performance in various domains. Additionally, it analyzes algorithm optimization strategies, including Double-Q-learning, Proximal Policy Optimization (PPO), MuZero, and deep evolutionary strategies (DES), comparing their effectiveness in solving complex tasks. Future research directions include the combination of optimization methods, with a focus on generalization and interpretability, as well as exploring real-world applications to further improve existing strategies. This review provides valuable insights for researchers and practitioners aiming to advance DRL technologies