A Deep Q-Learning Framework for Modeling and Nonlinear Adaptive Control of Autonomous Space Rocket Maneuvers

Abstract

In this paper, we develop a mathematical model and a deep Q-learning-assisted adaptive controller for autonomous space-rocket maneuvers. Our approach focuses on executing a multi-phase trajectory alignment without relying on external markers or inter-rocket communication. The proposed feedback mechanism uses onboard sensors to track the rocket's current position and orientation, applying standard robotics nomenclature for translational and rotational states. We formulate a general kinematic description of the rocket system, treating the maneuver as a tracking problem with respect to polynomial-based reference paths, which are generated online for each phase. A deep Q-learning module refines the control policy in real time by exploring actions that optimize the rocket's flight profile under uncertainties, such as unknown velocities. Simulation results verify that the integrated controller is capable of accurately following the desired trajectories and can robustly adapt to dynamic variations, illustrating the effectiveness of our proposed method for autonomous rocket guidance.