In this paper, the octahedral model of an athlete during physical training is reconstructed in real time by means of sensor wearing and data acquisition to recognize the changes of his/her body parts during movement. Two pose description methods, Euler angle and quaternion, are used to calculate the athlete’s actual motion position and the sensor reference position to locate the human motion position with high accuracy. Combine the occlusion processing method (DTP) and the action reconstruction algorithm (PE-DLS) to classify and complete the action reconstruction for different acceleration changes. The transformed action data are used to construct the action recognition model VT-AGCN, which accurately recognizes the actions in physical training. The athletes of the swimming team assisted by virtual reality technology have a greater improvement in stroke performance and leg striking performance, and the stroke performance is accelerated to 138.705-160.933 s, and the leg striking performance is accelerated to 190.158-208.846 s. Athletic training in the realm of competitive sports represents a vital factor for the development of superior athletes, and the employment of virtual reality technologies will be of great value in increasing the level of competitive ability of athletes.