收藏切换
Dougong detection method of Mask R-CNN Swin with an integrated attention mechanism
收藏切换
Yuan Jin1, Fang’ai Chi2, Ying’ao Liu1, Yan’e Li1, *
Journal of Image and Graphics | 2026, 31(8) : 2958 - 2972
Less
收藏切换
Journal of Image and Graphics | 2026, 31(8): 2958-2972
• Image Analysis and Recognition •
Dougong detection method of Mask R-CNN Swin with an integrated attention mechanism
Full
Yuan Jin1, Fang’ai Chi2, Ying’ao Liu1, Yan’e Li1, *
Affiliations
  • 1 College of Mathematics and Computer Science,Zhejiang A&F University,Hangzhou 311300,China
  • 2 School of Urban Construction,Yangtze University,Jingzhou 434000,China
Published: 2026-08-16 doi: 10.11834/jig.250460
Outline
收藏切换
Objective

Dougong is a distinctive structural and decorative component of traditional Chinese wooden architecture,playing a critical role in load transfer,seismic resistance,and architectural aesthetics. Moreover,Dougong,a key element connecting columns,beams,and roof structures,embodies sophisticated construction wisdom and regional architectural characteristics accumulated over centuries. The structural configuration of Dougong is composed of multiple interlocking wooden elements arranged in a highly ordered and hierarchical manner,exhibiting strong geometric regularity and pronounced directional dependency. These characteristics make Dougong not only architecturally significant but also visually complex. In real-world heritage environments,the automated recognition of Dougong faces substantial challenges. Variations in spatial orientation,component location (corner and non-corner Dougong),viewing angle,and scale significantly affect visual appearance. Moreover,complex backgrounds,uneven illumination,occlusion by surrounding architectural elements,and long-term material degradation further complicate accurate recognition. In particular,damaged Dougong components often exhibit incomplete structures and blurred boundaries,which severely reduce the effectiveness of conventional image recognition methods. Although deep learning techniques have been increasingly applied to cultural heritage documentation and architectural analysis,numerous existing approaches primarily rely on convolutional neural networks. Such methods often struggle to capture long-range dependencies and direction-sensitive structural features,limiting their adaptability to ancient architectural components with strong directional organization. Accordingly,a recognition framework that can explicitly model directional structural information while maintaining robustness under complex environmental conditions must be developed. The objective of this study is to develop a robust and accurate Dougong recognition model by integrating Transformer-based feature extraction with a novel direction-aware attention mechanism. This study aims to advance the intelligent recognition of intact and damaged Dougong components by enhancing directional feature representation and channel-wise feature interaction,thereby supporting the digital preservation and systematic documentation of ancient wooden architectural heritage.

Method

This study proposes an improved Dougong recognition framework based on the Mask R-CNN architecture enhanced with a Swin Transformer backbone. Unlike the traditional convolutional backbones,the Swin Transformer enables hierarchical feature extraction and efficient modeling of global contextual information,which is particularly beneficial for complex architectural scenes. A dedicated Dougong dataset is constructed,consisting of 3 014 carefully annotated images that include corner and non-corner Dougong instances. The dataset is collected from real architectural environments at historically significant sites,such as Lingyin Temple and Faxisi Temple,complemented by high-quality images from publicly accessible online repositories. The collected samples cover diverse viewpoints,spatial scales,lighting conditions,and background environments,ensuring strong representativeness and practical applicability. A directional coordinate attention (DIRCA) mechanism is introduced to address the limitations of conventional attention mechanisms in capturing the directional structural characteristics of Dougong. Unlike traditional spatial or channel attention methods that treat spatial dimensions uniformly,DIRCA decomposes spatial attention into horizontal and vertical directions. This design enables the network to independently encode direction-sensitive information along orthogonal axes,which closely corresponds to the intrinsic structural logic of Dougong,including component stacking order,load transmission paths,and repetitive decorative patterns. DIRCA improves the model’s ability to distinguish subtle structural variations under complex background interference by explicitly strengthening the representation of direction-dependent features. In addition,a 1D squeeze-and-excitation (1D-SE) module is incorporated to achieve lightweight channel recalibration. The proposed 1D-SE mechanism models inter-channel dependencies using a simplified 1D operation,effectively minimizing computational cost while preserving channel discrimination capability. This lightweight design facilitates seamless integration with the Transformer-based backbone and enhances the interaction between global contextual features and local structural details. The DIRCA and 1D-SE modules are jointly embedded within the Swin Transformer-enhanced Mask R-CNN framework,forming the proposed SwinE_AM_DIRCA model. Model training and evaluation are conducted under consistent experimental settings. Comparative experiments are performed against representative baseline models,including Mask R-CNN_ResNet50 and Swin_AM,to verify the effectiveness of the proposed architectural improvements. Multiple evaluation metrics are used to assess recognition performance,stability,and robustness,with particular emphasis on corner Dougong and damaged Dougong scenarios.

Result

Experimental results demonstrate that the proposed SwinE_AM_DIRCA model achieves substantial and consistent performance improvements over baseline methods across multiple recognition tasks. The recognition performance for non-corner Dougong reaches 94.4%,while the performance for corner Dougong attains 89.2%,outperforming the Swin_AM model by 6.7% and 8.9%,respectively. The proposed framework achieves even more pronounced improvements of 10.8% for non-corner Dougong and 20.4% for corner Dougong recognition compared with the conventional Mask R-CNN_ResNet50 model,highlighting the effectiveness of Transformer-based feature extraction combined with direction-aware attention. In damaged Dougong recognition tasks,the proposed model exhibits strong robustness and stability. Despite challenges,such as partial occlusion,surface erosion,structural incompleteness,and background clutter commonly observed in ancient architectural heritage sites,SwinE_AM_DIRCA maintains accurate localization and recognition of the key structural components. Qualitative visualization results further indicate that the model effectively suppresses irrelevant background features while preserving the integrity of direction-sensitive structural information,which is crucial for a reliable heritage documentation. Ablation experiments further confirm the contribution of each proposed module. The DIRCA mechanism significantly enhances the model’s sensitivity to directional features,while the 1D-SE module improves channel-wise feature discrimination with minimal computational overhead. When these two modules are combined,they produce complementary effects,resulting in consistent performance gains and improved model robustness across different Dougong types and damage conditions.

Conclusion

This study presents a direction-aware and lightweight attention-enhanced deep learning framework for Dougong recognition in ancient Chinese wooden architecture. The proposed SwinE_AM_DIRCA model effectively addresses the challenges posed by complex morphology,strong directional characteristics,and real-world damage conditions of Dougong structures by integrating the DIRCA mechanism and 1D-SE module into a Swin Transformer-based Mask R-CNN architecture. The primary innovation of this study lies in the unified modeling of direction-sensitive structural features and efficient channel recalibration,enabling improved recognition performance and robustness without excessive computational complexity. Experimental results demonstrate that the proposed method significantly outperforms existing baseline models in standard and damaged Dougong recognition tasks. Overall,this study provides an effective and practical technical solution for the intelligent recognition and digital preservation of ancient architectural components. The proposed framework contributes to the advancement of deep learning applications in cultural heritage analysis and offers valuable methodological support for the systematic documentation and sustainable protection of ancient buildings.

DIRCA  /  deep learning  /  cultural heritage  /  Transformer  /  ancient building
金远, 池方爱, 刘迎奥, 李颜娥. Mask R-CNN Swin融合注意力机制的斗拱检测方法. 中国图象图形学报, 2026 , 31 (8) : 2958 -2972 . DOI: 10.11834/jig.250460
Yuan Jin, Fang’ai Chi, Ying’ao Liu, Yan’e Li. Dougong detection method of Mask R-CNN Swin with an integrated attention mechanism[J]. Journal of Image and Graphics, 2026 , 31 (8) : 2958 -2972 . DOI: 10.11834/jig.250460
  • Natural Science Foundation of Zhejiang Province,China (MS26F030005,LQ21H180001);Research Development Foundation of Zhejiang A&F University (2019RF065)
Year 2026 volume 31 Issue 8
18
9
Cite this Article
BibTeX
Article Info
doi: 10.11834/jig.250460
  • Receive Date:2025-10-12
  • Online Date:2026-10-30
  • Published:2026-08-16
Article Data
Affiliations
History
  • Received:2025-10-12
  • Revised:2026-01-04
Funding
Natural Science Foundation of Zhejiang Province,China (MS26F030005,LQ21H180001);Research Development Foundation of Zhejiang A&F University (2019RF065)
Affiliations
    1 College of Mathematics and Computer Science,Zhejiang A&F University,Hangzhou 311300,China
    2 School of Urban Construction,Yangtze University,Jingzhou 434000,China
References
Share
https://castjournals.cast.org.cn/joweb/zgtxtxxb/EN/10.11834/jig.250460
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT