In the distribution inspection scene, UAV images are continuously used to obtain defect information on the surface of insulators, fittings, wire clips and conductors. Such areas need to quickly complete level identification and service on-site maintenance scheduling. The consistency of manual interpretation is insufficient, and the staged processing is easy to cause feature fragmentation. To this end, this paper proposes a hierarchical attention guided end-to-end UAV inspection image distribution defect intelligent grading framework. A dataset containing 6240 labeled images and four defect levels is constructed, and multi-scale enhancement is used to alleviate the uneven distribution of samples. In the defect representation stage, hierarchical attention is used to jointly encode local texture, edge changes and structural context. In the grading stage, the end-to-end mapping network synchronously completes feature aggregation and grade prediction. The experimental results show that the proposed method achieves 93.4% classification accuracy, 92.1% macro-F1 and 41 ms average inference time for a single image. The framework can provide stable application support for defect grading and auxiliary inspection of distribution components under complex aerial photography conditions.