收藏切换
Brain structure-function coupling hypergraph neural network
收藏切换
Mengqi Lei, Xiangmin Han, Siqi Li, Yue Gao*
Journal of Image and Graphics | 2026, 31(7) : 2526 - 2537
Less
收藏切换
Journal of Image and Graphics | 2026, 31(7): 2526-2537
• Foundation Models for Medical Image Analysis •
Brain structure-function coupling hypergraph neural network
Full
Mengqi Lei, Xiangmin Han, Siqi Li, Yue Gao*
Affiliations
  • School of Software,Tsinghua University,Beijing 100084,China
Published: 2026-07-16 doi: 10.11834/jig.250535
Outline
收藏切换
Objective

In recent years,brain networks have become an indispensable cornerstone in brain disorder research. At present,functional magnetic resonance imaging (fMRI) is commonly used to construct functional networks,while diffusion tensor imaging (DTI) is utilized to build structural networks. However,two limitations remain prominent. First,many approaches still operate on a single modality or adopt shallow multimodal fusion,leaving the potential dependency between structural connectivity (SC) and functional connectivity (FC) under-explored. Second,although structure-function coupling (SFC) has been demonstrated to differ across brain regions between healthy controls and patients,its relationship to classification has not been systematically integrated into representation learning to hold biomarker potential. This study aims to operationalize SFC as an explicit prior that guides multimodal representation learning,such that high-order cross-modal associations can be identified and translated into robust diagnostic performance.

Method

This study presents SFC-hypergraph neural network (HGNN),an SFC-guided foundational framework that combines a dual-stream encoder,cross-modal reconstruction pretraining,and hypergraph computation to model high-order relationships inside and between SC and FC. The core design is to use an SFC matrix as a bridge that explicitly guides two HGNN branches—one per modality—such that each branch learns modality-appropriate group relations while remaining informed by the other modality. Concretely,the SFC matrix is first constructed to capture the consistency between FC and SC patterns across brain regions by using a rank-based correlation measure. This matrix is fused with each modality through shallow feature mapping to form initial node-level features for the two streams. On the functional stream,hyperedges are formed via the sparse representation method,providing data-driven,high-order groupings that are appropriate for FC. On the structural stream,hyperedges are built with the k-nearest neighbor strategy to reflect global structural associations. Then,each branch employs HGNN layers to realize high-order messages that are passing on its respective hypergraph,encoding latent dependency between SC and FC under SFC guidance. To fully utilize cross-modal dependency,a cross-modal reconstruction pretraining task is introduced. During pretraining,the functional stream is trained to reconstruct the SC matrix while the structural stream reconstructs the FC matrix via decoders. The decoders are optimized with a reconstruction objective combined with a constraint that emphasizes the appropriate symmetry and sparsity of the reconstructed matrices. This pretraining forces the encoder to internalize modality-bridging information under SFC guidance. In the subsequent downstream tuning phase,the encoder parameters are frozen. Then,latent node representations from the two branches are flattened and concatenated into a global feature vector,which is fed into a lightweight multilayer perceptron classifier that is optimized with cross-entropy. Freezing the encoder keeps downstream training simple and stable while preserving the cross-modal dependency captured during pretraining.

Result

Experiments are conducted on two public multimodal brain imaging datasets. On the Alzheimer’s disease neuroimaging initiative (ADNI),we use 332 subjects,specifically,64 Alzheimer’s Disease (AD),129 mild cognitive impairment (MCI),and 139 normal control (NC). On the autism brain imaging data exchange (ABIDE),we include 86 subjects with paired fMRI and DTI for autism spectrum disorder (ASD)/NC. We adopt the automated anatomical labeling atlas for both datasets. fMRI data are preprocessed with the data processing assistant for resting-state fMRI and DTI data with the pipeline for analyzing brain diffusion images. The model is implemented in PyTorch and trained on an NVIDIA RTX 4090 GPU. We follow a two-stage training protocol (cross-modal pretraining followed by frozen-encoder tuning) and evaluate with fivefold cross-validation,reporting accuracy (ACC),area under the curve (AUC),F1,and specificity. We compare SFC-HGNN against representative single-modality baselines (BrainNetCNN,GAT,HGNN+,BrainGNN) and multi-modality methods (MME-GCN,Cross-GNN). Across diagnostic tasks,SFC-HGNN achieves state-of-the-art performance,consistently improving ACC,AUC,and F1. Although certain baselines occasionally yield higher specificity,these cases are frequently accompanied by markedly lower F1,suggesting reduced stability. By contrast,SFC-HGNN maintains better overall balance among metrics and demonstrates stronger robustness. Ablation studies further underscore the contributions of the proposed components: 1) introducing SFC as a cross-modal bridge (without pretraining) improves accuracy by 1.6% and 2.3% on AD versus NC and ASD versus NC,respectively,and 2) adding cross-modal reconstruction pretraining brings additional gains of 2.1% and 1.9%. These results indicate that SFC guidance,along with cross-modal reconstruction,effectively encourages the encoder to capture latent SC-FC dependency that transfers to downstream classification. Finally,we conduct significant hyperedge analysis to assess interpretability by using group-level t-tests and visualize discriminative functional and structural hyperedges for AD versus NC and ASD versus NC. The identified regions and connections align with known patterns of network alteration,supporting the neurobiological plausibility of the learned high-order representations.

Conclusion

By utilizing SFC as an explicit bridge and pairing dual HGNN encoders with a cross-modal reconstruction pretraining paradigm,this study introduces a principled multimodal framework for brain network analysis and brain disorder diagnosis. The approach encourages each modality to be informed by the other and encode high-order associations that are useful for downstream classification,while a frozen encoder tuning strategy keeps optimization stable and lightweight. On the ADNI and ABIDE datasets,SFC-HGNN consistently surpasses single-modal and multimodal baselines,with gains reflected not only in ACC and AUC but also in better balance between F1 and specificity,highlighting robustness. The significant hyperedge findings further provide neurobiologically plausible insights that complement the quantitative improvements. Overall,SFC-HGNN advances multimodal brain disorder diagnosis by unifying SFC-guided hypergraph encoding,cross-reconstruction pretraining,and simple downstream tuning into a coherent pipeline that achieves superior and reliable performance across datasets.

brain disorder diagnosis  /  multimodal brain network  /  structure-function coupling(SFC)  /  hypergraph neural network(HGNN)  /  cross-modal reconstruction  /  brain network foundation models
雷孟奇, 韩向敏, 李思奇, 高跃. 脑结构—功能耦合超图神经网络. 中国图象图形学报, 2026 , 31 (7) : 2526 -2537 . DOI: 10.11834/jig.250535
Mengqi Lei, Xiangmin Han, Siqi Li, Yue Gao. Brain structure-function coupling hypergraph neural network[J]. Journal of Image and Graphics, 2026 , 31 (7) : 2526 -2537 . DOI: 10.11834/jig.250535
  • National Science and Technology Major Project (2025ZD0217300);National Natural Science Foundation of China (62501358)
Year 2026 volume 31 Issue 7
26
13
Cite this Article
BibTeX
Article Info
doi: 10.11834/jig.250535
  • Receive Date:2025-10-30
  • Online Date:2026-10-30
  • Published:2026-07-16
Article Data
Affiliations
History
  • Received:2025-10-30
  • Revised:2026-03-09
Funding
National Science and Technology Major Project (2025ZD0217300);National Natural Science Foundation of China (62501358)
Affiliations
    School of Software,Tsinghua University,Beijing 100084,China
References
Share
https://castjournals.cast.org.cn/joweb/zgtxtxxb/EN/10.11834/jig.250535
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT