Semantic segmentation is a fundamental task in computer vision that aims to assign a semantic label to each pixel in an image. Most conventional semantic segmentation methods are designed for a closed-set setting in which the categories involved in testing are predefined and largely consistent with those observed during training. Although this setting facilitates the learning of stable category representations and clear decision boundaries,it limits the applicability of segmentation models in real-world scenarios,where unseen objects and novel semantic concepts are frequently encountered. In this context,open-vocabulary semantic segmentation (OVSS) has emerged as an important research direction for extending semantic segmentation from closed-set prediction to open-world visual understanding. OVSS enables models to identify and segment pixel-level regions corresponding to arbitrary textual concepts by using natural language descriptions as category definitions,thereby alleviating the dependence on fixed label spaces. The rapid development of vision-language pre-trained models,especially contrastive language-image pre-training (CLIP) and related large-scale cross-modal models,has provided an important foundation for OVSS. Trained on massive image-text pairs,these models learn aligned visual and textual representations and exhibit strong transferability in open-category recognition. However,most existing vision-language models are mainly optimized for image-level semantic alignment and global visual recognition. Thus,these models have difficulty meeting the requirements of pixel-level dense prediction directly,such as accurate localization,boundary delineation,and fine-grained category discrimination. Therefore,how to effectively transfer image-level open-vocabulary recognition ability to pixel-level segmentation remains a key challenge in OVSS. This paper summarizes recent progress in OVSS from the perspectives of task background,representative methods,benchmark datasets,evaluation metrics,remote sensing extension,and future directions. First,the paper introduces the task background and basic characteristics of OVSS,after which its relationship with conventional semantic segmentation and zero-shot semantic segmentation is clarified. Compared with conventional semantic segmentation,OVSS emphasizes generalization beyond predefined categories,whereas when compared with zero-shot semantic segmentation,it further benefits from large-scale image-text pre-training and allows more flexible category definitions through natural language descriptions. Second,representative OVSS methods are reviewed in accordance with several major research routes,including zero-shot semantic segmentation,early explorations based on image-text supervision,two-stage methods,and single-stage methods. Early zero-shot segmentation methods mainly rely on semantic embeddings to transfer knowledge from seen categories to unseen categories,but their expressive ability is often limited by static semantic representations. Meanwhile,initial studies based on image-text supervision explore whether transferable semantic region representations can be learned without dense pixel annotations,thus providing important inspiration for subsequent OVSS methods. Two-stage methods usually decompose the task into class-agnostic region generation and open-vocabulary region recognition. While this paradigm is clear and modular,its performance is highly dependent on the quality of candidate regions and may incur additional computational cost. In contrast,single-stage methods aim to integrate region modeling,vision-language alignment,and pixel-level prediction within a unified framework,thereby reducing cross-stage error accumulation and improving inference efficiency. Nevertheless,they still face challenges in fine-grained localization,dense semantic alignment,and robust generalization to unseen categories. Apart from analyzing the main ideas,technical characteristics,advantages,and limitations of these methods,this paper discusses the extension of OVSS to remote sensing imagery. Compared with natural images,remote sensing images typically exhibit top-down perspectives,large variations in object scale,significant orientation changes,complex backgrounds,and cross-region distribution shifts,all of which make it difficult to directly transfer OVSS methods designed for natural-image scenarios to remote sensing applications. Therefore,recent studies have begun to explore open-vocabulary remote sensing segmentation frameworks,dedicated benchmark datasets,training-free strategies,and multimodal fusion methods. This development indicates that OVSS is gradually expanding from general natural-image understanding to more complex specialized visual domains. Commonly used datasets and evaluation metrics in OVSS are also summarized in this paper. Existing studies typically use COCO-Stuff as the main training benchmark and evaluate model generalization on datasets,such as Pascal VOC,Pascal Context,and ADE20K,under different vocabulary settings. At present,OVSS evaluation relies on conventional semantic segmentation metrics,especially mean intersection over union (mIoU). However,such metrics emphasize strict category matching and are insufficient for measuring semantic similarity,category hierarchy,and open-category generalization. Therefore,constructing evaluation protocols that better reflect the semantic openness of OVSS remains an important issue. Finally,the paper summarizes the major challenges in current OVSS research and discusses promising future directions,including the effective transfer of open-vocabulary recognition capabilities to precise pixel-level prediction,adaptive region generation,evaluation protocols tailored to open-vocabulary settings,learning with limited supervision,training-free inference,and extensions to domain-specific applications.
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |