Public speaking teaching is not only related to the completion quality of students ‘oral expression, but also directly affects their self-presentation ability in classroom, academic communication and professional scenes. Aiming at the problems of single evaluation dimension, lagging process feedback and difficult to continuously identify ability changes in traditional teaching, this paper takes the public speaking course of colleges and universities as the object, constructs a teaching effect analysis framework integrating questionnaire, voice, video and performance data. Python was used to complete multi-source data preprocessing, standardization, paired sample t-test and multi-modal comprehensive evaluation. The results showed that after 12 weeks of teaching intervention, the total score of students ‘self-confidence increased from 3.21 to 3.86, the comprehensive evaluation score increased from 70.84 to 81.27, and the public speaking test score increased from 72.64 to 84.38. The relevant changes showed that public speaking teaching had a significant promotion effect on the formation of students ‘self-confidence, the improvement of expression fluency and the enhancement of field control ability. The significance of this paper is that the introduction of computer supported multimodal analysis into public speaking course evaluation provides a more detailed evidence base for teaching diagnosis and ability cultivation.