Aiming at the problems of insufficient identification of students ‘differences, lagging exercise load adjustment and low teaching feedback accuracy in college physical education courses, an adaptive teaching strategy optimization method of smart physical education courses driven by reinforcement learning was constructed. The system fuses physical monitoring, sports performance, classroom participation and learning feedback data to form a dynamic state representation of students, and uses DDPG algorithm to optimize teaching content, exercise load and feedback methods. The experimental results show that the accuracy of model state recognition reaches 94.6%, the load safety control rate is 96.2%, the comprehensive achievement degree of the course is improved to 92.4%, and the comprehensive satisfaction degree is 93.2%. The results show that this method can improve the individual adaptability and intelligent regulation effect of college physical education curriculum.