Salient object detection (SOD) seeks to emulate human visual attention by locating and segmenting the most conspicuous objects in visual scenes. While deep-learning-based 2D RGB and RGB-D SOD methods have advanced substantially,360° omnidirectional (panoramic) images introduce unique challenges,most notably spherical-to-plane projection distortions (e.g.,equal rectangular projection(ERP) artifacts),severe polar-region deformation,and boundary discontinuities across the left-right wraparound. At the same time,depth cues are known to improve geometric reasoning in conventional RGB-D SOD,but their effective exploitation for 360° SOD is underexplored due to modality-alignment difficulties,noise in estimated depth,and the lack of fusion architectures tailored to panoramic projection effects. The objective of this work is to investigate and demonstrate how carefully designed cross-modal modeling and projection-aware feature processing can harness depth information to substantially improve salient object detection on ERP panoramic images,while remaining robust to depth estimation noise and projection artifacts.
We propose an asymmetric dual-branch U-Net architecture for RGB-D 360° SOD that explicitly accounts for ERP projection properties and for cross-modal alignment. The model contains three novel components: 1) a panoramic-aware receptive field module (PA-RFM) that enhances direction-sensitive context modeling via a longitude-latitude-global triplet attention scheme and a ring-padding/cropping strategy to respect horizontal periodicity; 2) an attention-guided fusion module (AFM) that performs adaptive,dynamic weighting of RGB and depth channels using combined channel and spatial attention plus a lightweight modality weight generator to suppress noisy depth responses while preserving boundary cues; and 3) a cross-modal guided collaborative decoder in which a fusion-centric decoding path is explicitly guided by an independent high-resolution RGB decoding path via attentional gating,yielding superior edge and texture recovery. The RGB encoder is a pretrained SAM2 Hiera backbone with lightweight adapters to limit finetuning cost; the depth encoder is a ResNet-34 modified for single-channel input. PA-RFM employs a four-branch multiscale dilated design with a panoramic context aggregation module (PCAM) at each branch to merge longitude attention (via circular padding + horizontal convolutions),latitude attention (via latitude-weighted vertical convolutions that attenuate polar exaggeration),and global semantic attention (via latitude-weighted global average pooling (GAP) and channel recalibration). Training is performed in PyTorch on NVIDIA RTX 3090 hardware using AdamW (lr = 1 × 10-3,weight decay = 5 × 10-4),cosine annealing to 1 × 10-7,batch size 4,up to 50 epochs,input resolution 512 × 1 024,and a structured loss composed of weighted BCE + weighted IoU with multilayer deep supervision.
We evaluate the proposed model on two public panoramic SOD benchmarks augmented with depth maps produced by a state-of-the-art 360° depth estimator: 360-SOD (500 ERP images; 400 train/100 test) and 360-SSOD (1 105 ERP images; 850 train/255 test). Quantitatively,our method achieves MAE = 0.015 1,max-F = 0.838 8,mean-F = 0.815 0,max-E = 0.933 1,mean-E = 0.924 0,and Sm = 0.873 6 on 360-SOD (same scores on 360-SSOD),outperforming 10 representative RGB-D 2D SOD methods and seven representative RGB 360° SOD methods in objective metrics and visual quality. Relative to the strongest competing baseline,our mean absolute error(MAE) is reduced by 13.7%,max-F increases by 4.64%,mean-F by 3.24%,and Sm by 1.96%,demonstrating substantial gains particularly in boundary accuracy and structural consistency. Extensive qualitative comparisons show our model is markedly better at detecting objects in severely distorted polar regions,maintaining left-right boundary continuity,and recovering fine edges in cluttered or low-contrast scenes. Ablation studies confirm the contribution of each module: PA-RFM yields the largest single-module improvement (vs. RFB and a no-PA-RFM baseline),PCAM’s three attention branches (longitude,latitude,global) act synergistically,and a ring-padding width of k = 2 provides the best tradeoff between boundary continuity and redundancy. AFM outperforms simple fusion strategies (add,concat,multiply) and several state-of-the-art (SOTA) fusion blocks (MobileSal,HIDANet,CPNet variants). The cross-modal guided decoder meaningfully improves edge/detail recovery over a variant that omits the RGB decoding path. We also test robustness to different depth estimators and find consistent improvements over RGB-only baselines; performance varies modestly with depth source,with Joint_360depth yielding the best overall results but not causing catastrophic degradation when replaced.
This study demonstrates that depth information can be effectively and robustly exploited for 360° omnidirectional SOD when combined with projection-aware receptive field design and modality-aware fusion and decoding strategies. The proposed PA-RFM addresses ERP-specific distortions (horizontal periodicity and polar exaggeration) via circular padding,latitude weighting and global recalibration; AFM adaptively reconciles noisy depth with RGB structure; and the cross-modal guided collaborative decoder preserves high-resolution RGB priors to restore fine boundaries and textures lost in fused representations. Together,these innovations lead to consistent and remarkable gains across standard panoramic SOD benchmarks,especially in polar regions and at projection boundaries. The approach is general and can be extended to other panoramic vision tasks and to setups with sensor-captured depth; future work will explore domain adaptation to real depth sensors,multiprojection fusion (ERP + CMP),and model compression for mobile/AR/VR deployment.
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |