收藏切换
Co-salient object detection with automatic semantic discovery
收藏切换
Huachao Chen, Yingzhou Chen, Bin Wang, Feng Wang, Jia Zhao*
Journal of Image and Graphics | 2026, 31(7) : 2645 - 2659
Less
收藏切换
Journal of Image and Graphics | 2026, 31(7): 2645-2659
• Image Analysis and Recognition •
Co-salient object detection with automatic semantic discovery
Full
Huachao Chen, Yingzhou Chen, Bin Wang, Feng Wang, Jia Zhao*
Affiliations
  • College of Computer and Information Engineering,Fuyang Normal University,Fuyang 236037,China
Published: 2026-07-16 doi: 10.11834/jig.250377
Outline
收藏切换
Objective

Co-salient object detection (Co-SOD) aims to identify target regions from a set of related images that are not only visually salient but also semantically consistent across the group. The core challenge of Co-SOD lies in detecting visually prominent objects within individual images and determining which objects exhibit cross-image consistency among multiple images. Traditional Co-SOD methods mainly rely on low-level visual features to construct inter-image relationships. These approaches extract discriminative representations on the basis of handcrafted features,such as color,shape,and texture,to mine common salient objects. Although such methods perform reasonably well under simple backgrounds,they often fail in complex real-world scenarios with strong background clutter,severe occlusions,or multiple coexisting objects. In these cases,non-co-salient but visually salient objects can easily interfere with detection,leading to high false detection rates,poor robustness,and limited generalization. In recent years,with the rapid progress of deep neural networks,Co-SOD methods that are based on convolutional neural networks or vision Transformer have been proposed. By learning high-level semantic features across images,these methods substantially improve detection performance. However,they typically still rely on modeling image groups as the basis of detection without sufficiently incorporating explicit semantic priors. As a result,they lack the capability for proactive understanding of image categories or user-specific targets and therefore struggle in open-domain,multiobject scenarios or when fine-grained semantic distinctions are required. Moreover,when an image contains multiple visually salient but semantically irrelevant objects,these models often fail to distinguish them correctly,mistakenly including non-co-salient objects in the detection results,thus reducing the overall performance. For instance,in multiclass image datasets,such as CoCA,a single image often contains multiple visually prominent objects,but only a subset of them is semantically related and constitute co-salient targets. Other salient regions,although visually striking,act as semantic distractors. Without explicit semantic filtering mechanisms,traditional methods are often unable to distinguish which objects are the true co-salient targets,thus decreasing the accuracy of Co-SOD. To address these limitations,this study proposes a semantic-guided open-vocabulary Co-SOD framework. The proposed method leverages language prompts to guide the model’s attention toward target regions that are semantically aligned with image categories,thereby enabling semantic filtering of co-salient objects in complex multiobject scenarios and improving detection accuracy and semantic generalization ability.

Method

The proposed semantic-guided open-vocabulary Co-SOD framework integrates three key components,namely,a CLIP-based group-consistent category generation method,the open-vocabulary object detection model OWLv2,and the universal segmentation model SAM,forming a complete pipeline that begins from semantic prompt generation to object detection and pixel-level segmentation. First,the system introduces a CLIP-based group-consistent category generation method to automatically derive the most representative group-consistent category for a given image set. This procedure eliminates the need for manually specifying class names,reduces human effort,and provides accurate and stable semantic prompts for subsequent detection. Specifically,the method computes cross-modal similarities between the visual features of each image in the group and candidate category text features and automatically generates the most representative category name that is consistent across the group to serve as a reliable input for the detection model. Second,the generated group-consistent category name is used as a natural language prompt and fed into the open-vocabulary object detection model OWLv2. OWLv2,which is pretrained on large-scale image-text pairs,possesses strong cross-modal semantic alignment and matching capabilities. By mapping image region and text prompt features into a unified semantic space,OWLv2 can identify target regions that are highly relevant to the natural language prompts. To further improve the robustness of semantic understanding,this work constructs multiple semantically related natural language phrases for each category name and adopts a multiprompt parallel input mechanism to feed them into OWLv2 simultaneously. This process enhances the model’s capacity to capture diverse semantic descriptions,improves stability and robustness in semantic matching,and effectively addresses uncertainties caused by language ambiguity. During the candidate region generation stage,similarities between region feature and text semantic vectors are calculated to identify candidate regions that are highly related to the input prompts,with the model outputting corresponding bounding boxes and confidence scores. A dynamic thresholding strategy is introduced to further improve detection flexibility and adaptability. This mechanism automatically adjusts detection thresholds on the basis of the confidence levels of semantic matches,thereby increasing the recall of low-confidence targets while maintaining precision. As a result,the framework can effectively detect occluded or weakly salient targets and avoid missed detections caused by overly strict thresholds. At the saliency segmentation stage,the bounding box regions filtered during the semantic detection stage are passed to segment anything model (SAM). SAM then performs pixel-level mask generation for each region of interest. Leveraging its powerful structural perception and boundary modeling capabilities,SAM produces high-quality segmentation results. The final system outputs salient masks that are structurally complete and have clear boundaries. This design effectively decouples the two subtasks of semantic perception and visual detail modeling,allowing each module to fully leverage its strengths,thereby enhancing the overall semantic guidance ability and object delineation quality of the system.

Result

Experiments are conducted on three publicly available datasets: CoCA,CoSal2015,and CoSOD3k. The proposed method is compared with 15 state-of-the-art Co-SOD methods. On the CoCA dataset,our method achieves a 25.7% improvement in F measure and a 5.2% reduction in the mean absolute error (MAE) compared with the second-best model. On the CoSOD3k dataset,our method improves the F measure by 2.5%. On the CoSal2015 dataset,it improves the F measure by 2.2% and reduces MAE by 1.2%. These results comprehensively validate the strong ability of the proposed framework to recognize semantically relevant co-salient objects in multiobject open scenarios. Ablation experiments conducted on the CoSOD3k dataset further confirm the effectiveness of the proposed approach in enhancing Co-SOD performance.

Conclusion

This study proposes a semantic-guided open-vocabulary Co-SOD framework. The framework leverages a CLIP-based group-consistent category generation method to automatically generate semantic prompts for image groups and integrates OWLv2 for open-vocabulary object detection with SAM for universal segmentation. This process enables explicit semantic filtering of co-salient targets and fine-grained pixel-level segmentation. The proposed framework substantially enhances semantic generalization and target recognition robustness,enabling the effective localization of semantically consistent salient objects in multiobject and complex background scenarios. By unifying three critical components,namely,semantic prompt generation,open-vocabulary detection,and saliency segmentation,this framework provides an innovative and practical solution for advancing Co-SOD in open environments.

co-salient object detection(Co-SOD)  /  semantic guidance  /  open vocabulary  /  co-salient object detection framework  /  cross-modal similarity  /  dynamic threshold adaptation strategy
陈华超, 陈颍州, 王斌, 王峰, 赵佳. 结合自动语义发现的协同显著性目标检测. 中国图象图形学报, 2026 , 31 (7) : 2645 -2659 . DOI: 10.11834/jig.250377
Huachao Chen, Yingzhou Chen, Bin Wang, Feng Wang, Jia Zhao. Co-salient object detection with automatic semantic discovery[J]. Journal of Image and Graphics, 2026 , 31 (7) : 2645 -2659 . DOI: 10.11834/jig.250377
  • National Natural Science Foundation of China(61906044);Natural Science Foundation of Anhui Province,China (2408085MF154);Key Natural Science Research Projects of Anhui Provincial Department of Education(2023AH050406,2023AH050418)
Year 2026 volume 31 Issue 7
23
11
Cite this Article
BibTeX
Article Info
doi: 10.11834/jig.250377
  • Receive Date:2025-08-26
  • Online Date:2026-10-30
  • Published:2026-07-16
Article Data
Affiliations
History
  • Received:2025-08-26
  • Revised:2025-11-19
Funding
National Natural Science Foundation of China(61906044);Natural Science Foundation of Anhui Province,China (2408085MF154);Key Natural Science Research Projects of Anhui Provincial Department of Education(2023AH050406,2023AH050418)
Affiliations
    College of Computer and Information Engineering,Fuyang Normal University,Fuyang 236037,China
References
Share
https://castjournals.cast.org.cn/joweb/zgtxtxxb/EN/10.11834/jig.250377
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT