In order to enhance the interactive experience of garden landscape design, this study constructs a mobile landscape interactive experience system that combines virtual reality and augmented reality technologies to explore the cognitive behaviors of users in the virtual environment. The system is based on AR-VR hybrid technology for landscape display, which innovates on the traditional garden touring method and adopts an interactive approach to allow visitors to interchange between virtual reality and augmented reality in real time. The system introduces a bi-directional long and short-term memory network, which enables the system to collect both past and future information from visitors, recognize the user’s gesture movements, and provide a scientific basis for the visual optimization of the garden. The study shows that the system’s prediction accuracy of tourists’ gesture movements is up to 98%. The image factor has the greatest influence on the overall user impression of the garden, with an explanation rate of 54.458%. The user experience test results of the system are good, and the scores of the system indicators are all greater than 4, which verifies the feasibility of the system in this paper in landscape interaction and garden optimization design. This paper provides referable ideas for the research of intelligent garden design.