Rural cultural memory is not only an important construction of rural cultural history and tradition, but also a link of cultural inheritance, and rural ancient architecture as an important material carrier of rural history and culture, as well as the memory space of rural culture and spirit, has a unique value. The “exogenous” drawings of ancient rural buildings (the exogenous drawings of ancient buildings in this study refer to the visual information system of ancient buildings, such as traditional boundary paintings, illustrations showing the construction techniques of ancient buildings, examples of ancient structural engineering, and landscape representations of the main body of ancient buildings). It records and contains rich rural cultural and historical pictorial information. Through archaeological analysis of historical images and case comparisons, we attempt to construct an interpretive framework to decode the layers of historical information in the pictographs. The study takes rural cultural memory as the perspective, and the inheritance of “exogenous drawing” aesthetics of ancient rural buildings extends the structure and creatively revitalizes the cultural genes. At the same time, under the intervention of the development of digital technology, the study explores the multidimensional social and cultural values of the exogenous drawings of ancient rural buildings and the various possible forms of rural cultural narratives. The research focuses on the retention and development of rural cultural memory and the transformation of its bearing forms, aiming to build a memory ecosystem for the development of humanistic rural cultural protection in the new era and provide an effective path for rural cultural protection.