计算机辅助设计与图形学学报

基于解耦表征学习的生成式视觉图像理解

PDF下载

蔡江海 ¹^,²⁾ , 黄成泉 ¹^,²^,³⁾^,^* , 王顺霞 ²⁾ , 杨贵燕 ²⁾ , 罗森艳 ²⁾ , 周丽华 ²⁾

计算机辅助设计与图形学学报 | 图像视频处理 2025,37(12): 2202-2212

收起

计算机辅助设计与图形学学报 |图像视频处理 2025 , 37 (12) : 2202 -2212

基于解耦表征学习的生成式视觉图像理解

全屏

蔡江海¹^,²⁾, 黄成泉¹^,²^,³⁾^,^*, 王顺霞²⁾, 杨贵燕²⁾, 罗森艳²⁾, 周丽华²⁾

作者信息

¹⁾贵州省模式识别与智能系统重点实验室　贵阳　550025

²⁾贵州民族大学数据科学与信息工程学院　贵阳　550025

³⁾贵州民族大学工程技术人才实践训练中心　贵阳　550025

通讯作者:

（hcq@gzmu.edu.cn）

作者简介:

蔡江海（1999—），男，硕士研究生，主要研究方向为计算机视觉与模式识别、解耦表征学习

黄成泉（1976—），男，博士，教授，硕士生导师，论文通信作者，主要研究方向为计算机图形学、计算机应用技术、深度学习；

王顺霞（1999—），女，硕士研究生，主要研究方向为图像处理

杨贵燕（1997—），女，硕士研究生，主要研究方向为图像处理

罗森艳（1999—），女，硕士研究生，主要研究方向为图像处理

周丽华（1983—），女，硕士，副教授，主要研究方向为深度学习、图像处理.

Generative Visual Image Understanding Based on Disentangled Representation Learning

Jianghai Cai¹^,²⁾, Chengquan Huang¹^,²^,³⁾^,^*, Shunxia Wang²⁾, Guiyan Yang²⁾, Senyan Luo²⁾, Lihua Zhou²⁾

Affiliations

¹⁾Key Laboratory of Pattern Recognition and Intelligent Systems of Guizhou Province, Guiyang　550025

²⁾College of Data Science and Information Engineering, Guizhou Minzu University, Guiyang　550025

³⁾Engineering Training Center, Guizhou Minzu University, Guiyang　550025

出版时间: 2025-12-20 doi: 10.3724/SP.J.1089.2024-00003

文章导航

摘要

收起

学习可解释的视觉图像表征以揭示图像变化因素是计算机视觉领域的研究热点. 现有的许多解耦方法通过使用额外的正则项发现图像变化因素并学习解耦表征，但通常导致解耦和生成质量之间的不平衡，影响视觉图像理解效果. 为此，从图像的可解释性变化出发，提出基于解耦表征学习的生成式视觉图像理解方法. 首先设计预先训练的Glow生成模型，获取目标图像的潜在表征；然后由潜在表征构建基于图像变化的学习策略，得到候选遍历的可解释方向；最后在对比学习视角下设计对比模块，根据候选遍历的可解释方向模拟图像变化，进而提取解耦表征. 在解耦领域流行的数据集Shapes3D，MPI3D，Anime，MNIST和Cars3D上的实验结果表明，所提方法取得较好的效果，其中，在Cars3D数据集上的MIG，DCI，FactorVAE score和β-VAE score指标值分别达到0.16，0.27，0.89和0.98，验证了该方法的有效性和可行性.

关键词

解耦表征学习 / 可解释方向 / 图像生成 / 对比学习

Abstract

收起

Interpretable visual image representation learning to reveal image variation factors is a hot research topic in computer vision. Many existing disentanglement methods discover variation factors of images and learn disentangled representations by using extra regularization term. However, it usually leads to an imbalance between disentanglement and generative quality, which affects visual image understanding. To address this issue, a generative visual image understanding method based on disentangled representation learning is proposed in terms of interpretable variations in images. Firstly, a pre-trained Glow model is designed to acquire the latent representations of target images. Secondly, a learning strategy based on image variation is constructed from the latent representations to obtain interpretable directions of candidate traversals. Finally, the contrast module is designed under the contrastive learning perspective to simulate image variations based on the interpretable directions of candidate traversals and then extract disentangled representations. The experimental results show that better results are achieved on the popular disentanglement datasets, which are Shapes3D, MPI3D, Anime, MNIST and Cars3D, where the MIG, DCI, FactorVAE score and β-VAE score metrics reach 0.16, 0.27, 0.89 and 0.98, respectively, on the Cars3D dataset, verifying the effectiveness and feasibility of the proposed method.

Key words

disentangled representation learning / interpretable direction / image generation / contrastive learning

引用本文

蔡江海, 黄成泉, 王顺霞, 杨贵燕, 罗森艳, 周丽华. 基于解耦表征学习的生成式视觉图像理解. 计算机辅助设计与图形学学报, 2025 , 37 (12) : 2202 -2212 . DOI: 10.3724/SP.J.1089.2024-00003

Jianghai Cai, Chengquan Huang, Shunxia Wang, Guiyan Yang, Senyan Luo, Lihua Zhou. Generative Visual Image Understanding Based on Disentangled Representation Learning[J]. Journal of Computer-Aided Design & Computer Graphics, 2025 , 37 (12) : 2202 -2212 . DOI: 10.3724/SP.J.1089.2024-00003

基金

收起

国家自然科学基金(62062024)
贵州省科技计划(黔科合基础-ZK[2021]一般342)
贵州省研究生教育教学改革重点项目(黔教合YJSJGKT[2021]018)
贵州省教育厅自然科学研究项目(黔教技[2022]015)
贵州省模式识别与智能系统重点实验室2022年度开放课题(GZMUKL[2022]KF03)

参考文献

收起

文献

收起

参考文献引证文献

排序方式：

[1]

Kim H, Mnih A. Disentangling by factorising[C]//Proceedings of the 35th International Conference on Machine Learning. New York: ICML, 2018: 2654-2663

[2]

Lopez R, Regier J, Jordan M I, et al. Information constraints on auto-encoding variational Bayes[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. New York: ACM Press, 2018: 6117-6128

[3]

Karras T, Laine S, Aila T. A style-based generator architecture for generative adversarial networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2019: 4396-4405

[4]

Chen R T Q, Li X C, Grosse R B, et al. Isolating sources of disentanglement in VAEs[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. New York: ACM Press, 2018: 2615-2625

[5]

Lin Z N, Thekumparampil K K, Fanti G, et al. InfoGAN-CR and modelcentrality: self-supervised model training and selection for disentangling GANs[C]//Proceedings of the 37th International Conference on Machine Learning. New York: ACM Press, 2020: Article No.569

[6]

Li Zhixin, Zheng Yongzhe, Zhang Canlong, et al. Combining deep feature and multi-label classification for semantic image annotation[J]. Journal of Computer-Aided Design & Computer Graphics, 2018, 30(2): 318-326 (in Chinese) (李志欣, 郑永哲, 张灿龙, 等. 结合深度特征与多标记分类的图像语义标注[J]. 计算机辅助设计与图形学学报, 2018, 30(2): 318-326)

[7]

Chen X, Duan Y, Houthooft R, et al. InfoGAN: interpretable representation learning by information maximizing generative adversarial nets[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. New York: ACM Press, 2016: 2180-2188

[8]

Xiao T H, Hong J P, Ma J W. DNA-GAN: learning disentangled representations from multi-attribute images[OL]. [2024-01-02]. https://arxiv.org/abs/1711.05415

[9]

Higgins I, Matthey L, Pal A, et al. Beta-VAE: learning basic visual concepts with a constrained variational framework[OL]. [2024-01-02]. https://openreview.net/forum?id=Sy2fzU9gl

[10]

Yi Z L, Chen Z Q, Cai H, et al. BSD-GAN: branched generative adversarial network for scale-disentangled representation learning and image synthesis[J]. IEEE Transactions on Image Processing, 2020, 29: 9073-9083

[11]

Shen Y J, Gu J J, Tang X O, et al. Interpreting the latent space of GANs for semantic face editing[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2020: 9240-9249

[12]

Dalva Y, Pehlivan H, Hatipoglu O I, et al. Image-to-image translation with disentangled latent vectors for face editing[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(12): 14777-14788

[13]

Endo Y. User-controllable latent transformer for StyleGAN image layout editing[J]. Computer Graphics Forum, 2022, 41(7): 395-406

[14]

Xu Y Y, Du Y, Xiao W P, et al. From continuity to editability: inverting GANs with consecutive images[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society Press, 2021: 13890-13898

[15]

Mo S T, Sun Z, Li C. Representation disentanglement in generative models with contrastive learning[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Los Alamitos: IEEE Computer Society Press, 2023: 1531-1540

[16]

He K M, Fan H Q, Wu Y X, et al. Momentum contrast for unsupervised visual representation learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2020: 9726-9735

[17]

Le-Khac P H, Healy G, Smeaton A F. Contrastive representation learning: a framework and review[J]. IEEE Access, 2020, 8: 193907-193934

[18]

Ren X C, Yang T, Wang Y W, et al. Learning disentangled representation by exploiting pretrained generative models: a contrastive learning view[OL]. [2024-01-02]. https://arxiv.org/abs/2102.10543

[19]

Kingma D P, Dhariwal P. Glow: generative flow with invertible 1×1 convolutions[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. New York: ACM Press, 2018: 10236-10245

[20]

Voynov A, Babenko A. Unsupervised discovery of interpretable directions in the GAN latent space[C]//Proceedings of the 37th International Conference on Machine Learning. New York: ICML, 2020: 9786-9796

[21]

Wu Z R, Xiong Y J, Yu S X, et al. Unsupervised feature learning via non-parametric instance discrimination[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society Press, 2018: 3733-3742

[22]

Wen Zaidao, Wang Jiarui, Wang Xiaoxu, et al. A review of disentangled representation learning[J]. Acta Automatica Sinica, 2022, 48(2): 351-374 (in Chinese) (文载道, 王佳蕊, 王小旭, 等. 解耦表征学习综述[J]. 自动化学报, 2022, 48(2): 351-374)

[23]

Reed S, Zhang Y, Zhang Y T, et al. Deep visual analogy-making[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. New York: ACM Press, 2015: 1252-1260

[24]

Gondal M W, Wuthrich M, Miladinović Đ, et al. On the transfer of inductive bias from simulation to the real world: a new disentanglement dataset[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems. New York: ACM Press, 2019: Article No.1410

[25]

Elizabeth Rani G, Sakthimohan M, Abhigna Reddy G, et al. MNIST handwritten digit recognition using machine learning[C]//Proceedings of the 2nd International Conference on Advance Computing and Innovative Technologies in Engineering. Los Alamitos: IEEE Computer Society Press, 2022: 768-772

[26]

Kumar A, Sattigeri P, Balakrishnan A. Variational inference of disentangled latent concepts from unlabeled observations[OL]. [2024-01-02]. https://arxiv.org/abs/1711.00848

[27]

Khrulkov V, Mirvakhabova L, Oseledets I, et al. Disentangled representations from non-disentangled models[OL]. [2024-01-02]. https://arxiv.org/abs/2102.06204

2025年第37卷第12期

PDF下载

引用本文

BibTeX

文章信息

doi: 10.3724/SP.J.1089.2024-00003

接收时间：2024-01-02
首发时间：2026-10-10
出版时间：2025-12-20

补充材料

相关文章

文章信息

作者

出版历史

收稿日期：2024-01-02
修回日期：2024-07-12

基金

国家自然科学基金(62062024)

贵州省科技计划(黔科合基础-ZK[2021]一般342)

贵州省研究生教育教学改革重点项目(黔教合YJSJGKT[2021]018)

贵州省教育厅自然科学研究项目(黔教技[2022]015)

贵州省模式识别与智能系统重点实验室2022年度开放课题(GZMUKL[2022]KF03)

作者信息

¹⁾贵州省模式识别与智能系统重点实验室　贵阳　550025

²⁾贵州民族大学数据科学与信息工程学院　贵阳　550025

³⁾贵州民族大学工程技术人才实践训练中心　贵阳　550025

通讯作者:

（hcq@gzmu.edu.cn）

参考文献

分享链接

https://castjournals.cast.org.cn/joweb/jsjfzsj/CN/10.3724/SP.J.1089.2024-00003

分享至

全文二维码

扫描看全文

引用本文

BibTeX

本文的引用情况

2种不同金属材料的力学参数

科 Family	属数 Number of genus	种数 Number of species	占总种数比例 Percentage of total species (%)	属 Genus	种数 Number of species	占总种数比例 Percentage of total species (%)
鹅膏菌科Amanitaceae	2	11	5.26	鹅膏菌属 Amanita	10	4.78
小菇科 Mycenaceae	2	12	5.74	丝盖伞属 Inocybe	5	2.39
多孔菌科 Polyporaceae	8	14	6.70	蜡蘑属 Laccaria	5	2.39
红菇科 Russulaceae	3	23	11.00	小皮伞属 Marasmius	6	2.87
				小菇属 Mycena	11	5.26
				光柄菇属 Pluteus	5	2.39
				红菇属 Russula	17	8.13
				栓菌属 Trametes	5	2.39

关闭全屏

BibTeX
EndNote
RefWorks
TxT