A Survey of Safe Reinforcement Learning Methods in Robotics

College of Computer Science and Electronic Engineering, Hunan University, Changsha, China

xieyuen@hnu.edu.cn

Abstract

Reinforcement learning has achieved remarkable results in complex decision-making tasks such as robot control, but its deployment in real-world environments still faces safety challenges. Safe reinforcement learning aims to ensure that the behavior of the agent does not violate key safety constraints while exploring the environment and optimizing long-term rewards. This review paper aims to explore safe reinforcement learning methods and introduces three main types of safe reinforcement learning methods: control theory-based methods, formal method-based methods, and constrained optimization-based methods. After summarizing the advantages and disadvantages of these methods, this paper further analyzes their performance on the mainstream safe reinforcement learning framework Omnisafe. By citing existing comparative experimental results, the performance of different methods in multiple tasks is evaluated. It is found that the constrained optimization method can better improve task performance while ensuring safety due to its strong adaptability and scalability. Finally, this paper summarizes the current research status of safe reinforcement learning and points out the challenges that still exist and future development directions.