born in 2001. PhD candidate. Student member of CCF. His main research interest includes machine learning
born in 1990. PhD, lecturer. Member of CCF. His main research interest includes machine learning
born in 2003. Master candidate. His main research interests include machine vision and image recognition
born in 1997. PhD. Student member of CCF. His main research interests include machine learning and image recognition
born in 1980. PhD, professor. Senior member of CCF. His main research interests include machine learning and medical image processing
In recent years, large vision-language models (VLM) have demonstrated remarkable zero-shot prediction capabilities in image classification tasks. Existing VLM-based classification approaches typically rely on a set of predefined category labels or external textual knowledge bases as semantic prior to align and match visual content with language. However, in real-world applications, such predefined labels often fail to cover the vast and dynamic concept space of the open world, thereby limiting the adaptability and generalization ability of these models. To address this challenge, a more open-ended task, generative image classification (GIC), is introduced, in which no prior semantic labels are provided, and the target categories reside in an open-ended, potentially unbounded semantic space, substantially increasing the classification difficulty. To tackle the complexity of GIC, a granularity-guided multi-model collaboration framework is proposed, designed to fully leverage the strengths of VLM in both image understanding and text generation. In this framework, prompt templates are carefully crafted to guide VLM in generating candidate labels at varying levels of semantic granularity. Subsequently, the candidate label space is expanded by a language model that associates semantically related categories based on the generated image descriptions at each granularity level. Finally, the most relevant label is identified through a semantic similarity retrieval process in the embedding space. The effectiveness and superiority of the proposed method for the GIC task are validated through extensive experiments on multiple fine-grained image classification benchmarks.
| 科 Family | 属数 Number of genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) | 属 Genus | 种数 Number of species | 占总种数比例 Percentage of total species (%) |
|---|---|---|---|---|---|---|
| 鹅膏菌科Amanitaceae | 2 | 11 | 5.26 | 鹅膏菌属 Amanita | 10 | 4.78 |
| 小菇科 Mycenaceae | 2 | 12 | 5.74 | 丝盖伞属 Inocybe | 5 | 2.39 |
| 多孔菌科 Polyporaceae | 8 | 14 | 6.70 | 蜡蘑属 Laccaria | 5 | 2.39 |
| 红菇科 Russulaceae | 3 | 23 | 11.00 | 小皮伞属 Marasmius | 6 | 2.87 |
| 小菇属 Mycena | 11 | 5.26 | ||||
| 光柄菇属 Pluteus | 5 | 2.39 | ||||
| 红菇属 Russula | 17 | 8.13 | ||||
| 栓菌属 Trametes | 5 | 2.39 |