Adaptability of Deep Q-Networks in Autonomous Driving: Influences of Variable Obstacle Dynamics and Road Conditions

Viterbi School of Engineering, University of Southern California, 3650 McClintock Ave Los Angeles, CA, United States

^* Corresponding author: yuanjuns@usc.edu

Abstract

Autonomous vehicles (AVs) have garnered significant attention due to their potential for enhancing transportation accessibility, and traffic safety. Despite numerous advancements, current AV systems still face challenges in fully autonomous operation under real-world conditions. This study modifies the CarRacing-v2 environment from the OpenAI Gymnasium to simulate realistic road conditions with obstacles. Deep Q- Network (DQN) combined with Convolutional Neural Networks (CNN) is utilized for perception and path planning. The approach investigates the impact of varying obstacle sizes, shapes, movement speeds, and road surface conditions on the agent’s performance. The model is trained over 3000 episodes, with the evaluation focusing on cumulative rewards. The results show that the agent's performance is inversely related to obstacle size; the smaller the obstacle, the poorer the performance, which can be attributed to insufficient feature extraction. Irregular-shaped obstacles and reduced surface traction both caused a delayed learning response but will eventually achieve the desired reward. Variations in obstacle colors showed that red obstacles caused confusion due to similarities with red curbs of the road, indicating the model’s over-dependence on luminance. In general, the agent shows adaptability to various road conditions, but enhancements in perception are needed for complex visual cues.