Outline

Ingegneria Sismica

Ingegneria Sismica

Reinforcement learning-based state space dimensionality reduction and optimal control strategy design in robot navigation systems

Author(s): Huaqi Liu1
1School of Engineering, University of Glasgow G12 8QQ, Glasgow, United Kingdom
Liu, Huaqi . “Reinforcement learning-based state space dimensionality reduction and optimal control strategy design in robot navigation systems.” Ingegneria Sismica Volume 43 Issue 2: 1-16, doi:10.65102/is2026825.

Abstract

Aiming at the problems of difficult high-dimensional state space modeling and complex continuous control strategy optimization in robot navigation, this paper proposes a reinforcement learning framework that integrates topological dimensionality reduction, Radon feature extraction and deep deterministic policy gradient (DDPG). Firstly, the topological dimensionality reduction method is used to construct the rotational mapping graph (RMG) and the feature network (CN), which reduces the path planning complexity by 90%. Second, the Radon transform variant is designed to extract 24-dimensional normalized environment feature vectors to compress the dimensionality of sensory data. Finally, the OU noise equilibrium exploration-utilization is introduced based on the DDPG algorithm to learn a continuous speed control strategy on the fused state space. Simulation validation shows that the state-space dimensionality reduction model reduces the error in signal tracking by 63% and improves the convergence speed by 300%. The DDPG navigation strategy achieves an average reward of 567 in dynamic obstacle environments, exceeding the benchmark algorithm by 45.7%. Only 6.76 million training samples are required to reach 100% navigation success rate, less than 6% of that under SCF and CPDRL algorithms. The training time is 8.07h, and the convergence step size is 61,039 steps, which improves the efficiency by more than 40%. This framework provides an efficient solution for real-time autonomous navigation in complex environments.

Keywords
reinforcement learning; topology dimensionality reduction; Radon transform; DDPG; state space dimensionality reduction; robot navigation

Related Articles

Honghong Duan1, Yanhui Wang1
1School of Marxism, University of Sanya, Sanya 572022, Hainan, China
Cheng Shao1
1College of Economics and Management, Jiangxi University of Technology, Nanchang 330098, Jiangxi, China
Guosen Ma1
1School of Mathematics and Statistics, Hainan Normal University, Haikou 571158, Hainan, China
Chunyan Yi1, Shan Huang2
1School of Foreign Languages ,Guangxi Minzu Normal University, Chongzuo 532200, Guangxi, China
2Department of Development Planning, Guangxi Minzu Normal University, Chongzuo 532200, Guangxi, China