Outline

Ingegneria Sismica

Ingegneria Sismica

Reinforcement learning-based state space dimensionality reduction and optimal control strategy design in robot navigation systems

Author(s): Huaqi Liu1
1School of Engineering, University of Glasgow G12 8QQ, Glasgow, United Kingdom
Liu, Huaqi . “Reinforcement learning-based state space dimensionality reduction and optimal control strategy design in robot navigation systems.” Ingegneria Sismica Volume 43 Issue 2: 1-16, doi:10.65102/is2026825.

Abstract

Aiming at the problems of difficult high-dimensional state space modeling and complex continuous control strategy optimization in robot navigation, this paper proposes a reinforcement learning framework that integrates topological dimensionality reduction, Radon feature extraction and deep deterministic policy gradient (DDPG). Firstly, the topological dimensionality reduction method is used to construct the rotational mapping graph (RMG) and the feature network (CN), which reduces the path planning complexity by 90%. Second, the Radon transform variant is designed to extract 24-dimensional normalized environment feature vectors to compress the dimensionality of sensory data. Finally, the OU noise equilibrium exploration-utilization is introduced based on the DDPG algorithm to learn a continuous speed control strategy on the fused state space. Simulation validation shows that the state-space dimensionality reduction model reduces the error in signal tracking by 63% and improves the convergence speed by 300%. The DDPG navigation strategy achieves an average reward of 567 in dynamic obstacle environments, exceeding the benchmark algorithm by 45.7%. Only 6.76 million training samples are required to reach 100% navigation success rate, less than 6% of that under SCF and CPDRL algorithms. The training time is 8.07h, and the convergence step size is 61,039 steps, which improves the efficiency by more than 40%. This framework provides an efficient solution for real-time autonomous navigation in complex environments.

Keywords
reinforcement learning; topology dimensionality reduction; Radon transform; DDPG; state space dimensionality reduction; robot navigation

Related Articles

Haibo Tian1, Haipeng Niu1, Kangmin Ji1, Zhiheng Zhou1
1College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, Shaanxi, China
Chuanqi Lao1
1Economics and Management School of Sichuan Tourism College, Chendu 610100, Sichuan, China
Yujia Li1
1School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510640, China
Guodong Wu1, Chao Chen2, Bin Ni1
1Jiangsu Vocational Institute of Commerce, Nanjing 211168, Jiangsu, China
2College of Physical Education of Dalian University, Dalian116622, Liaoning, China
Jian Liu1, Qien Hou2, Tuo Zhang1
1Department of Mechanical and Electrical Engineering, Huaibei Vocational and Technical College, Huaibei, 235000, Anhui, China
2College of Applied Chemical Industry, Huaibei Vocational and Technical College, Huaibei 235000, Anhui, China