Deep Reinforcement Learning in Continuous Control: Advances and Challenges

School of Computer Science, Hangzhou Dianzi University, Hangzhou, China
Multiple media and web development, Fuzhou University, Fuzhou, China
International College, Chongqing University of Posts and Telecommunications, Chongqing, China

^* 2022214972@stu.cqupt.edu.cn

Abstract

With the widespread application of Deep Reinforcement Learning (DRL) in continuous control, its learning and decision-making capabilities in high-dimensional state spaces have brought new opportunities for intelligent control systems. This paper provides a systematic review of the research progress of DRL in continuous control tasks, covering representative algorithms from classic Deep Deterministic Policy Gradient (DDPG) and Twin Delayed DDPG (TD3) to the soft policy method Soft Actor-Critic (SAC), and further focuses on key breakthroughs in algorithm structure and sample efficiency in recent years. In the latest research, one class of methods effectively improves the accuracy and stability of policy generation by aligning the behavior of diffusion models with the Q function; another class of methods employs Euclidean data augmentation techniques to enhance data diversity and generalization performance during the training process; additional studies introduce successor features and concurrent policy combination mechanisms, significantly improving transfer efficiency and adaptability in multi-task learning. These innovative algorithms show great potential in alleviating training instability, enhancing sample utilization, and improving policy generalization capabilities. Looking ahead, DRL research in continuous control can further explore efficient exploration mechanisms, model prediction integration, multi-agent collaboration, and real-world deployment, laying the foundation for building reliable and adaptive intelligent control systems.