The detection of floating objects in rivers and lakes based on unmanned aerial vehicle (UAV) is the basis for automatic water surface monitoring and fine-grained environmental monitoring. Manual inspection and traditional image processing methods are limited by complex ripples, shore shadows, small target scales, and unstable imaging angles. This paper proposes a multi-source deep learning framework for intelligent recognition and classification of floating objects in UAV river patrol images. The proposed framework combines visible light images, multispectral cues, and spatial state encoding to enhance the boundary representation of floating objects against a reflected water background. The lightweight detection branch locates the suspected floating area, and the classification branch is used to distinguish the categories of floating objects such as plastic bottles, foam boards, branches and leaves, bags and algae. A dataset consisting of 18,420 UAV images and 73,600 annotated objects is constructed from different river, lake, and wetland scenes. Experimental results show that the model achieves 91.8% mAP, 94.2% classification accuracy, 92.7% F1-score, 42.6 FPS, and FLOPs of 15.6B, which supports stable river patrol deployment and online analysis tasks.