Outline

Ingegneria Sismica

Ingegneria Sismica

Reinforcement learning-based state space dimensionality reduction and optimal control strategy design in robot navigation systems

Author(s): Huaqi Liu1
1School of Engineering, University of Glasgow G12 8QQ, Glasgow, United Kingdom
Liu, Huaqi . “Reinforcement learning-based state space dimensionality reduction and optimal control strategy design in robot navigation systems.” Ingegneria Sismica Volume 43 Issue 2: 1-16, doi:10.65102/is2026825.

Abstract

Aiming at the problems of difficult high-dimensional state space modeling and complex continuous control strategy optimization in robot navigation, this paper proposes a reinforcement learning framework that integrates topological dimensionality reduction, Radon feature extraction and deep deterministic policy gradient (DDPG). Firstly, the topological dimensionality reduction method is used to construct the rotational mapping graph (RMG) and the feature network (CN), which reduces the path planning complexity by 90%. Second, the Radon transform variant is designed to extract 24-dimensional normalized environment feature vectors to compress the dimensionality of sensory data. Finally, the OU noise equilibrium exploration-utilization is introduced based on the DDPG algorithm to learn a continuous speed control strategy on the fused state space. Simulation validation shows that the state-space dimensionality reduction model reduces the error in signal tracking by 63% and improves the convergence speed by 300%. The DDPG navigation strategy achieves an average reward of 567 in dynamic obstacle environments, exceeding the benchmark algorithm by 45.7%. Only 6.76 million training samples are required to reach 100% navigation success rate, less than 6% of that under SCF and CPDRL algorithms. The training time is 8.07h, and the convergence step size is 61,039 steps, which improves the efficiency by more than 40%. This framework provides an efficient solution for real-time autonomous navigation in complex environments.

Keywords
reinforcement learning; topology dimensionality reduction; Radon transform; DDPG; state space dimensionality reduction; robot navigation

Related Articles

Qianwen Xiong1, Yuhong Chen1
1Guangzhou University of Chinese Medicine, School of Pharmaceutical Medicine, Guangzhou,Guangdong,China,510006
Zhihao Jiang1,2, Limi Chen1,2, Jing Yang1
1Hainan Vocational University of Science and Technology, Haikou 571126, China
2Institute for Mathematical Research, Universiti Putra Malaysia, Serdang 43400, Malaysia
Limi Chen1,2, Zhihao Jiang1,2, Jing Yang1
1Hainan Vocational University of Science and Technology, Haikou 571126, China
2Institute for Mathematical Research, Universiti Putra Malaysia, Serdang 43400, Malaysia
Hui Yuan1, Minjie Chai2, Siqing Xu1, Jinsong Li1, Jinwan Zheng1
1Electric Power Research Institute, State Grid Shanxi Electric Power Co., Ltd., Taiyuan, 030001, Shanxi, China
2Jincheng Power Supply Branch, State Grid Shanxi Electric Power Co., Ltd., Jincheng, 048000, Shanxi, China
Yanhan Zhu1,2
1China Academy of Cultural Heritage, Chaoyang District, 100029, Beijing, China
2Beijing University of Civil Engineering and Architecture, Xicheng District, 100044, Beijing, China