In this paper, machine vision technology with non-contact, high efficiency and automation is selected for bridge structure measurement, expecting to build a practical real-time monitoring method. Combined with the relationship between the camera coordinate system and the world coordinate system, the camera calibration is carried out to eliminate the error caused by image distortion and realize the correction of image distortion. On this basis, the SIFT feature point detection method is selected to extract and match the image features to form the image measurement algorithm based on machine vision. The finite element structure is built based on the location information of the key points of the bridge acquired by vision, and the overall stress distribution data of the bridge structure is calculated. Meanwhile, based on the on-site measurement data, the three-dimensional deformation data of the bridge structure is output to complete the measurement of the bridge deflection. In the practical application of machine vision-based stress calculation and deflection measurement method in River Gorge Bridge K, the maximum stress calculation of the bridge cantilever top plate is 14.18MPa and the maximum stress calculation of the bottom plate is 17.49MPa, which is close to the actual measurement results, and the difference between the deflection measurement of the bridge and the GPS measurement is less than 1mm, which shows a wide range of prospects for the application of stress and deflection monitoring on bridges.