Outline

Ingegneria Sismica

Ingegneria Sismica

Multimodal AI-Based Semantic Annotation and Knowledge Graph Construction for the Motif of Withered Trees, Bamboo, and Rocks in Literati Painting

Author(s): Jikang Cao1
1School of Art and Design, Jiangxi Institute of Technology, Nanchang 330098, Jiangxi, China
Cao, Jikang. “Multimodal AI-Based Semantic Annotation and Knowledge Graph Construction for the Motif of Withered Trees, Bamboo, and Rocks in Literati Painting.” Ingegneria Sismica Volume 43 Issue 2: 1-18, doi:10.65102/is2026766.

Abstract

This research puts forward a multi-modal artificial intelligence frame for semantic mark and knowledge graph building of the theme of dead trees, bamboos and stones in Chinese scholar painting. A motif-directional dataset called WTBR-LP is built from open museum pictures, metadata files, names, carved words, description words, and expert labels. The marks explanation system includes three layers: vision things, pen skill shape, and culture meaning. The put-forward model incorporates open-vocabulary visual location-finding, area cutting, vision-language expression, metadata embedding, and graph-neighbor restrictions to produce region-level multi-label notes. The annotation outcomes are further changed into WTBR-KG, which is a source-conscious knowledge graph that connects art works, people, visual motifs, brush stroke characteristics, inscriptions, collection origins, and cultural concepts. In the early-stage experiment configuration, our put-forward method obtains a Macro-F1 of 0.864, mAP@0.5 of 0.803, Hits@1 of 0.912, and triple accuracy of 0.923. The work flow that is assisted by AI cuts down the time of expert checking from 8.70 minutes to 2.05 minutes on each single artwork. The obtained results show that multimodal fusing and graph restriction can promote the accuracy, interpretability, and traceability of semantic marking for scholar painting.

Keywords
literati painting; withered trees bamboo and rocks; multimodal AI; semantic annotation; knowledge graph

Related Articles

Qianwen Xiong1, Yuhong Chen1
1Guangzhou University of Chinese Medicine, School of Pharmaceutical Medicine, Guangzhou,Guangdong,China,510006
Zhihao Jiang1,2, Limi Chen1,2, Jing Yang1
1Hainan Vocational University of Science and Technology, Haikou 571126, China
2Institute for Mathematical Research, Universiti Putra Malaysia, Serdang 43400, Malaysia
Limi Chen1,2, Zhihao Jiang1,2, Jing Yang1
1Hainan Vocational University of Science and Technology, Haikou 571126, China
2Institute for Mathematical Research, Universiti Putra Malaysia, Serdang 43400, Malaysia
Hui Yuan1, Minjie Chai2, Siqing Xu1, Jinsong Li1, Jinwan Zheng1
1Electric Power Research Institute, State Grid Shanxi Electric Power Co., Ltd., Taiyuan, 030001, Shanxi, China
2Jincheng Power Supply Branch, State Grid Shanxi Electric Power Co., Ltd., Jincheng, 048000, Shanxi, China
Yanhan Zhu1,2
1China Academy of Cultural Heritage, Chaoyang District, 100029, Beijing, China
2Beijing University of Civil Engineering and Architecture, Xicheng District, 100044, Beijing, China