收藏切换
Lightweight image super-resolution network with sparse and permuted self-attention
收藏切换
Siqi Wu, Wei Liu*, Weidong Chen
Journal of Image and Graphics | 2026, 31(8) : 2897 - 2910
Less
收藏切换
Journal of Image and Graphics | 2026, 31(8): 2897-2910
• Image Processing and Coding •
Lightweight image super-resolution network with sparse and permuted self-attention
Full
Siqi Wu, Wei Liu*, Weidong Chen
Affiliations
  • School of Computer Science,South China Normal University,Guangzhou 510631,China
Published: 2026-08-16 doi: 10.11834/jig.250519
Outline
收藏切换
Objective

Image super-resolution (SR) is a fundamental low-level vision task in computer vision that aims to reconstruct high-resolution images from low-resolution inputs,thereby enhancing image detail quality and sharpness. High-quality reconstruction not only improves human visual perception but also provides clear and highly structured input features for high-level vision tasks,such as object detection,semantic segmentation,and face recognition. Therefore,SR technology has broad application prospects in intelligent surveillance,medical imaging,autonomous driving,satellite remote sensing,and related fields. Convolutional neural networks (CNNs) have been widely used for SR because of their ability to capture local textures and edges. However,their limited receptive fields hinder the modeling of long-range dependencies,which are essential for preserving global structural consistency. Transformer-based methods,by contrast,leverage self-attention mechanisms to capture the global context,achieving superior reconstruction performance. Despite this advantage,conventional Transformers suffer from high computational costs and redundant attention weight,limiting their applicability in lightweight or real-time scenarios. To address these challenges,we propose a lightweight SR network that integrates the advantages of CNN and Transformer architectures. The network integrates efficient local feature extraction with global context modeling,achieving balance between reconstruction quality and computational efficiency.

Method

We propose SPSANet,a lightweight image SR network that is based on sparse and permuted self-attention (SPSA) and composed of shallow feature extraction,deep feature extraction,and high-resolution reconstruction. At the shallow feature extraction stage,a standard 3 × 3 convolution captures basic texture information,and shallow features are forwarded to the reconstruction stage via a long skip connection to provide residual guidance. The deep feature extraction stage consists of four SPSA groups,each containing six SPSA blocks and a convolutional layer. Each block integrates an SPSA module (SPSAM) that expands the receptive field while reducing attention redundancy,a high-frequency enhancement module (HFEM) for refining texture and edge details,and a dual-branch feature fusion module (DBFFM) that effectively fuses global and local features through spatial–channel interactions. Then,shallow and deep features are fused via a global residual connection,and a pixel shuffle operation generates the final high-resolution output. Our method is implemented on the PyTorch framework and trained using an NVIDIA RTX 3090 GPU. The training dataset is DIV2K,which contains 800 images. For the ×2 SR task,the model is trained for a total of 500 K iterations by using the Adam optimizer with an initial learning rate of 2 × 10⁻⁴. The input image patch size is fixed at 64 × 64,the batch size is set to 16,and the MultiStepLR scheduler is applied to halve the learning rate at iterations [250 K,400 K,450 K,475 K]. For the ×3 and ×4 SR tasks,the model is initialized with the pretrained weights from the ×2 model,and the total number of training iterations is reduced by half.

Result

The proposed method is evaluated on five public benchmark datasets and compared with 11 state-of-the-art (SOTA) SR approaches. Experimental results demonstrate that under the constraint of lightweight design,SPSANet achieves either the best or second-best performance across different upscale factors and datasets,showing strong generalization ability and stability. Specifically,when the upscale factor is ×3,SPSANet surpasses the latest SOTA methods by 0.15 dB and 0.25 dB in peak signal-to-noise ratio (PSNR) on Urban100 and Manga109 datasets,respectively,and achieves corresponding improvements in the structural similarity index measure (SSIM). Furthermore,when the self-ensemble strategy is applied during the testing phase,the model’s performance is further enhanced. Under the same scaling factor,PSNR increases by an additional 0.11 dB and 0.23 dB on Urban100 and Manga109,respectively,indicating that the proposed method maintains strong robustness under multiview inference. Ablation studies are conducted on SPSAM,HFEM,and DBFFM to further investigate the contribution of each component. The results show that removing any of these modules leads to a noticeable degradation in performance. SPSAM plays a key role in global dependency modeling and attention sparsification,HFEM effectively enhances texture restoration quality,and DBFFM substantially improves the fusion of global and local features. In terms of visual quality,SPSANet can reconstruct clear,natural,and sharp image textures,particularly in scenes with complex structures and abundant details. It preserves the structural consistency and perceptual realism of the original images more effectively than existing approaches do.

Conclusion

In this work,we propose SPSANet,a lightweight image SR network that integrates an SPSA mechanism to efficiently capture long-range dependencies while minimizing redundant computations. By establishing interlayer attention connections,SPSANet enables focused feature learning,ensuring that the network prioritizes tokens that are the most critical to the reconstruction process. Moreover,the proposed HFEM enhances the recovery of fine textures and structural details,and DBFFM effectively aligns and integrates global and local representations. Through extensive experiments,SPSANet demonstrates superior reconstruction performance and visual fidelity compared with existing lightweight SR approaches,especially in challenging scenarios with intricate patterns and rich textures. Despite these promising results,opportunities for further improvement remain. Future work could enhance model compactness and generalization through knowledge distillation,hybrid convolution-Transformer integration,and adaptation to broad low-level vision tasks (e.g.,image denoising and deblurring). Overall,SPSANet provides a practical,interpretable solution for lightweight Transformer-based image restoration,offering insights for the design of efficient and high-performing SR networks.

deep learning  /  image super-resolution  /  sparse and permuted self-attention (SPSA)  /  high-frequency information enhancement  /  dual-branch feature fusion (DBFF)
吴思琪, 柳薇, 陈卫东. 轻量级稀疏置换自注意力图像超分辨率网络. 中国图象图形学报, 2026 , 31 (8) : 2897 -2910 . DOI: 10.11834/jig.250519
Siqi Wu, Wei Liu, Weidong Chen. Lightweight image super-resolution network with sparse and permuted self-attention[J]. Journal of Image and Graphics, 2026 , 31 (8) : 2897 -2910 . DOI: 10.11834/jig.250519
Year 2026 volume 31 Issue 8
13
6
Cite this Article
BibTeX
Article Info
doi: 10.11834/jig.250519
  • Receive Date:2025-11-05
  • Online Date:2026-10-30
  • Published:2026-08-16
Article Data
Affiliations
History
  • Received:2025-11-05
  • Revised:2025-12-19
Affiliations
    School of Computer Science,South China Normal University,Guangzhou 510631,China
References
Share
https://castjournals.cast.org.cn/joweb/zgtxtxxb/EN/10.11834/jig.250519
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT