收藏切换
CloudPredUNet: a satellite cloud image prediction network incorporating frequency domain self-attention
收藏切换
Qi He1, Zhengyuan Zang1, 2, Zengzhou Hao2, *
Journal of Image and Graphics | 2026, 31(9) : 3412 - 3426
Less
收藏切换
Journal of Image and Graphics | 2026, 31(9): 3412-3426
• Remote Sensing Image Processing •
CloudPredUNet: a satellite cloud image prediction network incorporating frequency domain self-attention
Full
Qi He1, Zhengyuan Zang1, 2, Zengzhou Hao2, *
Affiliations
  • 1 College of Information Science,Shanghai Ocean University,Shanghai 201306,China
  • 2 State Key Laboratory of Satellite Ocean Environment Dynamics,Second Institute of Oceanography,Ministry of Natural Resources,Hangzhou 310012,China
Published: 2026-09-16 doi: 10.11834/jig.250559
Outline
收藏切换
Objective

Accurate short-term prediction of satellite cloud imagery is crucial for modern meteorological operations,including severe weather nowcasting,aviation safety,and disaster preparedness. However,this task remains profoundly challenging due to the inherently nonlinear and chaotic dynamics of atmospheric motion. Cloud systems undergo complex spatiotemporal transformations,such as genesis,dissipation,deformation,rotation,merging,and splitting,which are difficult to capture using conventional forecasting techniques. Traditional approaches,including optical flow and block-matching methods,primarily estimate displacement vectors based on local brightness constancy or regional correlation. While computationally manageable,these methods often fail to model the nonstationary and nonrigid transformations characteristic of real cloud evolution,leading to considerable error accumulation over extended prediction horizons.

Method

To overcome these challenges,this study proposes CloudPredUNet,a novel and lightweight neural network architecture for satellite cloud image sequence prediction. The model is built upon a U-Net skeleton,renowned for its effective encoder-decoder structure with skip connections,which is well-suited for dense prediction tasks. CloudPredUNet introduces three core innovative modules designed to enhance global feature aggregation,detail reconstruction,and long-range spatiotemporal dependency modeling,respectively. First,in the encoder pathway,we design a frequency domain self-attention (FDSA) module. Instead of applying global Fourier transforms,which can be sensitive to noise and computationally heavy for high-resolution images,our module operates on localized image patches. The input features are projected and split into query (Q),key (K),and value (V) components. The Q and K tensors are partitioned into nonoverlapping patches. A 2D fast Fourier transform (FFT) is applied to each patch,transitioning the representation into the frequency domain. A Hadamard (element-wise) product between the spectra of Q and K patches is then performed,effectively computing cross-correlation in the frequency domain,a process equivalent to spatial convolution but with considerably lower computational complexity (O(HW log (HW))). The result is transformed back to the spatial domain via inverse FFT,and then gated with the V component through element-wise multiplication. This design enables efficient aggregation of global contextual information across the entire image while preserving local texture details,addressing the limited receptive field of standard convolutions. Second,the decoder pathway incorporates a multiscale spatial attention (MSA) module to recover high-resolution details and suppress irrelevant background regions during upsampling. This module employs a series of cascaded depthwise separable convolutions with progressively increasing dilation rates (e.g.,d = 1,3,5,7). This multibranch structure captures features at multiple scales,mitigating the gridding artifacts common in single-dilation dilated convolutions. The outputs from all branches are concatenated and processed through a spatial attention mechanism. This mechanism generates an attention map by applying max-pooling and average-pooling across the channel dimension,followed by a small multilayer perceptron (MLP) and a sigmoid activation. This map highlights structurally important regions (e.g.,cloud edges,cores) and attenuates uniform areas,thereby refining the reconstruction quality. Third,a dedicated temporal mapper is introduced between the encoder and decoder to model the evolution of features over time explicitly. Its core component is a spatiotemporal feature extraction (SFE) module. This module first merges the time and channel dimensions of the encoded feature sequence. The merged features are then split into four subsets. One subset is preserved,while the other three are processed by distinct convolutional branches: a standard 3 × 3 depthwise convolution,a 1 × 11 horizontal strip convolution,and an 11 × 1 vertical strip convolution. This heterogeneous receptive field design allows the model to capture isotropic cloud expansion as well as anisotropic motion patterns (e.g.,shear,directional advection). The outputs are fused and further refined by a channel recalibration block,which employs channel-wise max and average pooling followed by an MLP to weight the importance of different temporal channels adaptively,enhancing the modeling of long-range dependencies. The complete CloudPredUNet model is trained end-to-end using the mean squared error (MSE) loss function,optimizing it to minimize pixel-wise prediction error directly.

Result

Extensive experiments were conducted on a dataset constructed from the FY-2D geostationary meteorological satellite,covering the region 26° E—146° E,60° S—60° N. The dataset was temporally and spatially partitioned to ensure no overlap between training,validation,and independent test sets. CloudPredUNet was evaluated against a comprehensive suite of state-of-the-art models,including traditional optical flow,autoregressive models (ConvLSTM,PredRNN,MIM),and nonautoregressive models (MMVP,SimVP,TAU). Results demonstrate the superior performance and remarkable efficiency of CloudPredUNet. In terms of model efficiency,CloudPredUNet is exceptionally lightweight,containing only 0.101 million parameters and requiring approximately five GFLOPs per forward pass. This represents a drastic reduction compared with competitors; for instance,its parameter count is less than one-fifth of the next most efficient model (MMVP),and its computational cost is nearly two orders of magnitude lower than ConvLSTM. Regarding prediction accuracy,CloudPredUNet achieved the best scores across all four primary evaluation metrics on the test set: MAE of 8.393,MSE of 188.922,SSIM of 0.825,and PSNR of 25.367. This consistent outperformance indicates its exceptional capability in minimizing prediction error while maintaining high structural fidelity and image quality relative to the ground truth. A critical test for any sequence prediction model is its performance in multistep forecasting. CloudPredUNet demonstrated outstanding robustness against error accumulation. In a five-frame-ahead prediction task,it secured the lowest MAE for every single predicted frame. Notably,by the fifth frame,its MAE (10.589) was approximately 2.4% lower than that of the second-best model,MIM (10.852),underscoring its superior stability for longer-term predictions. Ablation studies systematically validated the contribution of each proposed module. Removing any of the three core modules (FDSA,MSA,SFE) led to a measurable drop in performance. The complete model provided an 11.63% improvement in MAE over a baseline U-Net without these modules,confirming their effectiveness and complementary nature. Qualitative visual analysis further reinforced these quantitative findings. In complex meteorological scenarios such as cloud growth,cloud merging,and the preservation of a typhoon eye’s characteristic structure,CloudPredUNet generated predictions that were visually more coherent,detailed,and consistent with the actual evolution compared with other models. It successfully maintained sharp boundaries and realistic textures where other models produced blurring,fragmentation,or loss of critical features.

Conclusion

This study addresses key limitations in satellite cloud image prediction,namely,the modeling of nonlinear cloud dynamics,the efficient capture of global spatial dependencies,and the mitigation of multistep error accumulation,by introducing CloudPredUNet. The core innovations lie in its integrative design: The frequency domain self-attention module enables efficient global context modeling; the multiscale spatial attention module enhances detail reconstruction; and the temporal mapper with its spatiotemporal feature extraction module strengthens long-range evolutionary pattern learning. The experimental outcomes affirm that CloudPredUNet successfully bridges the gap between high predictive accuracy and operational efficiency. It establishes a new state-of-the-art on the FY-2D benchmark while being orders of magnitude more parameter- and compute-efficient than prevailing methods. This combination makes it a highly promising candidate for real-time,operational short-term cloud forecasting systems where computational resources may be constrained. Future research will focus on several avenues to advance this work toward greater practical utility and robustness,including the following: 1) expanding the spatiotemporal diversity of training and testing datasets to evaluate and improve the model’s generalizability across different seasons,climatic zones,and synoptic conditions; 2) investigating multimodal fusion frameworks that incorporate ancillary meteorological data fields (e.g.,wind vectors,humidity,atmospheric pressure) to inject more physical constraints into the learning process,potentially improving the prediction of physically coherent evolutions; and 3) further refining the frequency-domain processing mechanism,potentially by integrating adaptive filters or multiresolution analysis,to better handle high-frequency noise and further enhance the reconstruction of subtle cloud microstructures. Through these efforts,we aim to evolve CloudPredUNet from a high-performing research prototype into a reliable and versatile tool for operational meteorology.

satellite cloud image prediction  /  spatiotemporal prediction  /  FY-2D satellite  /  frequency domain self-attention  /  spatial attention
贺琪, 臧正源, 郝增周. 融合频域自注意力的卫星云图预测网络CloudPredUNet. 中国图象图形学报, 2026 , 31 (9) : 3412 -3426 . DOI: 10.11834/jig.250559
Qi He, Zhengyuan Zang, Zengzhou Hao. CloudPredUNet: a satellite cloud image prediction network incorporating frequency domain self-attention[J]. Journal of Image and Graphics, 2026 , 31 (9) : 3412 -3426 . DOI: 10.11834/jig.250559
  • Natural Science Foundation of Zhejiang Province,China(LZJMZ24D060001);National Natural Science Foundation of China(42376194)
Year 2026 volume 31 Issue 9
14
6
Cite this Article
BibTeX
Article Info
doi: 10.11834/jig.250559
  • Receive Date:2025-11-17
  • Online Date:2026-10-30
  • Published:2026-09-16
Article Data
Affiliations
History
  • Received:2025-11-17
  • Revised:2026-01-21
Funding
Natural Science Foundation of Zhejiang Province,China(LZJMZ24D060001);National Natural Science Foundation of China(42376194)
Affiliations
    1 College of Information Science,Shanghai Ocean University,Shanghai 201306,China
    2 State Key Laboratory of Satellite Ocean Environment Dynamics,Second Institute of Oceanography,Ministry of Natural Resources,Hangzhou 310012,China
References
Share
https://castjournals.cast.org.cn/joweb/zgtxtxxb/EN/10.11834/jig.250559
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT