Journal of Soft Computing and Information Technology

Journal of Soft Computing and Information Technology

A Hybrid Generalized Zero-Shot Learning Method for Image Classification

Document Type : Persian Original Article

Authors
Seyed Mosayeb Arabi
Amineh Amini
Hadi Saboohi
Department of Computer Engineering, Karaj Branch, Islamic Azad University, Karaj, Iran.
Abstract
Abstract- One of the main challenges in machine learning is the need for extensive training data, which increases the cost and time required for data preparation. In some cases, there may be limited data available, or privacy concerns may arise. Generalized zero-shot learning (GZSL) offers a suitable option in such situations, enabling the prediction of both seen and unseen classes. Although various methods have been proposed in this field, accurately recognizing seen and unseen classes still presents multiple challenges. This paper introduces a hybrid generalized zero-shot learning (H-GZSL) approach, which improved classification accuracy in GZSL. The proposed method simultaneously utilizes two classification techniques, enhancing classification accuracy for both seen and unseen classes. To address different scenarios and resolve conflicts in classification, a correction mechanism is employed. This mechanism compares the predicted classes from the two methods and assigns the instance to the most appropriate one. The proposed method was evaluated on the AWA2, SUN, aPY, CUB, and FLOW datasets. The evaluation separately measured the performance on seen and unseen classes, and the harmonic mean of classification accuracy across these classes was calculated. Compared to previous methods, the proposed approach achieved an improvement in the harmonic mean accuracy on the CUB and SUN datasets.
Keywords
Classification
Zero-shot learning
Generalized zero-shot learning
[1] X. Wang, A. Liu, L. Wu, C. Li, Y. Liu and X. Chen, “A Generalized Zero-Shot Learning Scheme for SSVEP-Based BCI System,” in IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 31, pp. 863-874, 2023.
[2] L. Niu, J. Cai, A. Veeraraghavan, and L. Zhang, “Zero-shot learning via category-specific visual-semantic mapping and label refinement”, IEEE Transactions on Image Processing, vol. 28, no. 2, pp. 965–979, 2018.
[3] ] H. Jiang, R. Wang, S. Shan, and X. Chen, “Learning class prototypes via structure alignment for zero-shot recognition,” in Proceedings of the European Conference on Computer Vision, 2018, pp. 121–138.
[4] Wei Wang, “Generalized Zero-Shot Activity Recognition with Embedding-Based Method”, ACM Transactions on Sensor Networks, 19, 1–25, , 2023.
[5] Qian Wang, “Generalized zero-shot domain adaptation via coupled conditional variational autoencoders”, Neural Networks, 163, 40-52, 2023.
[6] E. Schonfeld, S. Ebrahimi, S. Sinha, T. Darrell, and Z. Akata, “Generalized zero-and few-shot learning via aligned variational autoencoders,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 8247–8255.
[7] Y. Ye, Y. He, T. Pan, J. Li and H. T. Shen, “Alleviating Domain Shift via Discriminative Learning for Generalized Zero-Shot Learning, ” in IEEE Transactions on Multimedia, vol. 24, pp. 1325-1337, 2022.
[8] Y. An, S. Kim, Y. Liang, R. Zimmermann, D. Kim and J. Kim, “Content-Attribute Disentanglement for Generalized Zero-Shot Learning, ” in IEEE Access, vol. 10, pp. 58320-58331, 2022.
[9] A. Paul, N. C. Krishnan, and P. Munjal, “Semantically aligned bias reducing zero shot learning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 7056–7065.
[10] Richard Socher, “zero-shot learning through cross-modal transfer”, 2013.
[11] Wei-Lun Chao, “an empirical study and analysis of generalized zero-shot learning for object recognition in the wild”, 2017.
[12] Maxime Bucher, “Generating Visual Representations for Zero-Shot Classification”, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), 2017.
[13] Kibok Lee, “hierarchical novelty detection for visual object recognition”, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
[14] Yongqin Xian, “Feature Generating Networks for Zero-Shot Learning”, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
[15] Haofeng Zhang, “Triple Verification Network for Generalised Zero-shot Learning”, IEEE Transactions on Image Processing, 2019, 28, 506 – 517.
[16] Yashas Annadani, “Preserving Semantic Relations for Zero-Shot Learning”, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018.
[17] Pengkai Zhu, “Generalized Zero-Shot Recognition based on Visually Semantic Embedding”, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[18] Vinay Kumar Verma, “Towards Zero-Shot Learning with Fewer Seen Class Examples”, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), 2020.
[19] Lu Liu, “Attribute Propagation Network for Graph Zero-shot Learning”, AAAI Conference on Artificial Intelligence, 2020.
[20] Jie Shao, “Generalized Zero-Shot Learning With Multi-Channel Gaussian Mixture VAE”, IEEE Signal Processing Letters, 2020, 27, 456 – 460.
[21] Xiao Li, Min Fang, “Learning domain invariant unseen features for generalized zero-shot classification”, Knowledge-Based Systems, Volume 206, 2020.
[22] Tingting Xu, “Dual Generative Network with Discriminative Information for Generalized Zero-Shot Learning”, Complexity, 2021.
[23] Zhi Chen, “Entropy-Based Uncertainty Calibration for Generalized Zero-Shot Learning”, Databases Theory and Applications, 2021.
[24] Yang, Shiqi & Wang, Kai, “On Implicit Attribute Localization for Generalized Zero-Shot Learning”, IEEE Signal Processing Letters, PP. 1-1. 10.1109/LSP, 2021.
[25] J. Ding, X. Hu and X. Zhong, “A Semantic Encoding Out-of-Distribution Classifier for Generalized Zero-Shot Learning,” in IEEE Signal Processing Letters, vol. 28, pp. 1395-1399, 2021.
[26] L. Feng and C. Zhao, “Transfer Increment for Generalized Zero-Shot Learning,” in IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 6, pp. 2506-2520, June 2021.
[27] X. Chen, J. Li, X. Lan and N. Zheng, “Generalized Zero-Shot Learning Via Multi-Modal Aggregated Posterior Aligning Neural Network,” in IEEE Transactions on Multimedia, vol. 24, pp. 177-187, 2022.
[28] T. Shermin, S. W. Teng, F. Sohel, M. Murshed and G. Lu, “Bidirectional Mapping Coupled GAN for Generalized Zero-Shot Learning,” in IEEE Transactions on Image Processing, vol. 31, pp. 721-733, 2022.
[29] G. -S. Xie et al., “Generalized Zero-Shot Learning With Multiple Graph Adaptive Generative Networks,” in IEEE Transactions on Neural Networks and Learning Systems, vol. 33, no. 7, pp. 2903-2915, July 2022.
[30] ] H. Su, J. Li, K. Lu, L. Zhu and H. T. Shen, “Dual-Aligned Feature Confusion Alleviation for Generalized Zero-Shot Learning,” in IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 8, pp. 3774-3785, Aug. 2023.
[31] Z. Wang, Y. Gou, J. Li, L. Zhu and H. T. Shen, “Language-Augmented Pixel Embedding for Generalized Zero-Shot Learning,” in IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 3, pp. 1019-1030, March 2023.
[32] Y. Yang, X. Zhang, M. Yang and C. Deng, “Adaptive Bias-Aware Feature Generation for Generalized Zero-Shot Learning,” in IEEE Transactions on Multimedia, vol. 25, pp. 280-290, 2023.
[33] Y. Zhang, S. Feng, “Enhancing Domain-Invariant Parts for Generalized Zero-Shot Learning,” in Proceedings of the 31st ACM International Conference on Multimedia, pp. 6283–6291, 2023.
[34] Y. Ye, T. Pan, T. Luo, J. Li and H. T. Shen, “Learning MLatent Representations for Generalized Zero-Shot Learning,” in IEEE Transactions on Multimedia, vol. 25, pp. 2252-2265, 2023.
[35] J. Cavazza, V. Murino and A. D. Bue, “No Adversaries to Zero-Shot Learning: Distilling an Ensemble of Gaussian Feature Generators,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 10, pp. 12167-12178, Oct. 2023.
[36] J. Wu, T. Zhang, Z. -J. Zha, J. Luo, Y. Zhang and F. Wu, “Prototype-Augmented Self-Supervised Generative Network for Generalized Zero-Shot Learning, “ in IEEE Transactions on Image Processing, vol. 33, pp. 1938-1951, 2024.
[37] Liang, Y., Cao, W. Isolation and distillation network for generalized zero-shot learning. Neural Comput & Applic 36, 13935–13955, 2024.
[38] Farhad Pourpanah, “A Review of Generalized Zero-Shot Learning Methods, IEEE Transactions on Pattern Analysis and Machine Intelligence, 45, 4051 – 4070, 2023.
[39] Li, M. Ye, L. Zhou, D. Zhang, C. Zhu, and Y. Liu, “Improving generalized zero-shot learning by semantic discriminator,” arXiv preprint arXiv:2005.13956, 2020.
[40] R. Gao et al., “Zero-VAE-GAN: Generating Unseen Features for Generalized and Transductive Zero-Shot Learning,” in IEEE Transactions on Image Processing, vol. 29, pp. 3665-3680, 2020.
[41] Chen, Shiming, Hong, Ziming, Xie, Guo-Sen, Yang, Wenhan, Peng, Qinmu, Wang, Kai, Zhao, Jian and You, Xinge. “MSDN: Mutually Semantic Distillation Network for Zero-Shot Learning”, Paper presented at the meeting of the CVPR, 2022.
[42] Z. Han, Z. Fu, S. Chen and J. Yang, “Contrastive Embedding for Generalized Zero-Shot Learning,” in 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, 2021.
[43] Zhenqi Z, Cao W, “Visual-semantic consistency matching network for generalized zero-shot learning”, Neurocomputing 536:30–39, 2023.
[44] F. Zhang and G. Shi, “Co-representation network for generalized zero-shot learning,” in Proc. Int. Conf. Mach. Learn., pp. 7434–7443, 2019.
[45] Y. Yu, Z. Ji, J. Han, and Z. Zhang, “Episode-based prototype generating network for zero-shot learning,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2020, pp. 14032–14041.
Journal of Soft Computing and Information Technology
Volume 13, Issue 3
Autumn - December
Autumn 2024
Pages 46-59