U-structure has become a foundational approach in medical image segmentation, consistently demonstrating strong performance across various segmentation tasks. Most current models are based on this framework, customizing encoder–decoder components to achieve higher accuracy across various segmentation challenges. However, this often comes at the cost of increased parameter counts, which inevitably limit their practicality in real-world applications. In this study, we provide an E-shaped segmentation framework that discards the traditional step-by-step resolution recovery decoding process, instead directly aggregating multi-scale features extracted by the encoder at each stage for deep cross-level integration. Additionally, we propose an innovative multi-scale large-kernel convolution (MLKConv) module, designed to enhance high-level feature representation by effectively capturing both local and global contextual information. Compared to U-structure, the proposed E-structured approach substantially reduces parameters while delivering superior performance, especially in complex segmentation tasks. Based on this structure, we develop 2 segmentation networks specifically for 2-dimensional (2D) and 3D medical images. 2D E-SegNet is evaluated on four 2D segmentation benchmark datasets (Synapse multi-organ, ACDC, Kvasir-Seg, and BUSI), while 3D E-SegNet is assessed on four 3D segmentation benchmark datasets (Synapse, ACDC, NIH Pancreas, and Lung). Experimental results demonstrate that our approach outperforms the current leading U-shaped models across multiple datasets, achieving new state-of-the-art (SOTA) performance with fewer parameters. In summary, our research introduces a novel approach to medical image segmentation, offering potential improvements and contributing to ongoing advancements in the field. Our code is publicly available on https://github.com/zhaoqi106/E-SegNet.
| 1. | We introduce an E-shaped segmentation architecture that removes conventional step-by-step decoding process, differing from the U-structure and significantly reducing model parameters. A novel MLKConv module is designed, utilizing depthwise convolutions with varying scales and dilation rates to efficiently extract and fuse local and global information with minimal parameters. |
| 2. | Based on this E-structure, we construct an efficient 2D segmentation model and evaluate it on 4 different types of public medical datasets. Our model surpasses most current U-structure methods in segmentation accuracy and achieves the lower parameter count and faster inference speed among models with comparable performance. |
| 3. | Furthermore, we develop a 3D segmentation model based on this E-structure with ViT as the encoder, demonstrating the structure’s generality across 2D and 3D segmentation tasks. Testing on 4 public 3D medical segmentation datasets, we achieve leading results with fewer parameter count. |
| 4. | We re-design several U-shaped networks, adapting them to E-structure, and perform comparative analyses with their original U-structure counterparts. This evaluation validates the extensibility of E-structure by assessing its advantages and limitations. |
U-shaped models: A class of encoder–decoder architectures (e.g., U-Net) designed for image segmentation tasks, characterized by symmetric downsampling and upsampling paths with skip connections to recover spatial resolution. | |
CNN: Convolution-based neural networks are widely used in visual recognition and segmentation tasks, capable of learning hierarchical spatial features through stacked convolutional layers and local receptive fields. | |
ViT: A transformer-based architecture that models image patches as sequences, enables effective global context capture without relying on convolutional operations. | |
SOTA: Refers to the best performance or most advanced approach reported in the literature for a given task at the time of evaluation. | |
Encoder: The downsampling path of a segmentation model is used to extract increasingly abstract and semantic features by reducing spatial dimensions and increasing receptive fields. | |
Decoder: The upsampling path of a segmentation model is responsible for progressively recovering spatial resolution and reconstructing the segmentation mask from encoded features. | |
Skip connections: Lateral links between corresponding encoder and decoder stages in U-shaped architectures are designed to fuse low-level spatial features with high-level semantic information. | |
Cascade structure: A parallel multi-branch design within the encoder or decoder, where each branch models features in different ways (e.g., using different resolutions or network architectures), enables richer feature representation in medical image segmentation. | |
2D medical image segmentation: Segmentation tasks perform on individual image slices (e.g., CT or MRI), treating each 2D slice independently. | |
3D medical image segmentation: Volumetric segmentation tasks consider spatial context across multiple slices (e.g., using 3D convolution or attention across depth, height, and width dimensions). | |
FPS: Indicates the speed at which a model processes input images or volumes, commonly used to evaluate inference efficiency in real-time or clinical settings. | |
Five-fold cross-validation: A model validation strategy divides the dataset into 5 subsets, using 4 for training and 1 for testing in rotation, thereby improving the robustness of performance estimates. | |
Backbone: The core feature extractor in a deep learning model, often composes of pretrained architectures (e.g., ResNet and Swin Transformer), which generates multi-level feature representations for downstream tasks like segmentation. | |
Stem (stem layer): The initial convolutional block of a neural network is responsible for basic spatial feature extraction from input images, typically before entering deeper stages of the encoder. | |
Video Swin Transformer: A hierarchical transformer-based backbone is originally designed for video action recognition, adapted here to capture long-range dependencies and spatiotemporal context in 3D medical volumes via shifted window self-attention. | |
Stages of aggregation: Refers to the module where multi-scale features (from all encoder levels) are spatially aligned (e.g., by upsampling) and fused to produce enhanced representations for segmentation. In E-SegNet, this replaces traditional decoder stages. | |
MaxVit: Combines convolutional operations with both block-wise and grid-wise attention to effectively capture spatial dependencies across scales, offering a unified and efficient architecture for vision tasks. |
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |