A Supervised Method for Building a Regularized Map for General Multi-View Multi-Manifold Learning

Document Type : Persian Original Article

Authors

1 Faculty of computer and information technology engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran

2 Faculty of Computer and Information Technology Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran

3 Data scientist Advanced Analytics Department, General Motors, Warren, MI, USA

Abstract

In this paper, we consider the issue of automatic and unsupervised class-manifold selection in a multi-view multi-manifold space. General multi-manifold learning methods achieve multiple independent manifolds, so it is challenging for them to adjust the intra-class local manifold information and global inter-class discriminative structure. In this paper, we propose a multi-manifold embedding method, which can explicitly obtain multi-view multi-manifold structure while considering both intra-class compactness and inter-class separability without using the class label information. Furthermore, to the generalization of embedding to novel points, known as the out-of-sample extension problem in multi-view multi-manifold learning, we propose a supervised method for building a regularized map that provides an out-of-sample extension for general multi-view multi-manifold learning studied in the context of classification. Experimental results on face and object images demonstrate the potential of the proposed method for the classification of multi-view multi-manifold data sets and the proposed out-of-sample extension algorithm for the classification of manifold-modeled data sets.

Keywords

References

[1]     J. Li, Y. Wu, J. Zhao, and K. Lu, "Multi-manifold Sparse Graph Embedding for Multi‐modal Image Classification," Neurocomputing, vol. 173, pp. 501–510, 2016.
[2]     YangxiLi, XinShi, CuilanDu, YangLiu, and YonggangWen, "Manifold regularized multi-view feature selection for social image annotation," Neurocomputing, vol. 204, pp. 135-141, 2016.
[3]     S. Sun, "A survey of multi-view machine learning," Neural Comput & Applic, vol. 23, pp. 2031–2038, 2013.
[4]     B. Schölkopf, A.Smola, and K.-R.Müller, "Kernel principal component analysis," Artificial NeuralNetworks-ICANN'97,Springer,Lausanne,Switzerland,, pp. 583–588, 1997.
[5]     W. Liu, H. Zhang, D. Tao, Y. Wang, and K. Lu, "Large-scale paralleled sparse principal component analysis," Multimed.ToolsAppl, pp. 1-13, 2013.
[6]     W. Gao, B. Cao, S. Shan, X. Chen, D. Zhou, X. Zhang, et al., "The CAS-PEAL Large-Scale Chinese Face Database and Baseline Evaluations," IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART A: SYSTEMS AND HUMANS, vol. 38, pp. 149-161, 2008.
[7]     X. Geng, D. C. Zhan, and Z. H. Zhou, "Supervised Nonlinear Dimensionality Reduction for Visualization and Classification," IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, vol. 35, pp. 1098-1107, 2005.
[8]     B. Raducanu and F.Dornaika, "A supervised non-linear dimensionality reduction approach for manifold learning," Pattern Recognition, vol. 45, pp. 2432–2444, 2012.
[9]     F. Aeini, A. m. Eftekhari-Moghadam, and F. Mahmoudi, "Non Linear Dimensional Reduction Method based on Supervised Neighborhood Graph," presented at the 7th International Symposium on Telecommunications (IST'2014), Tehran, Iran, 2014.
[10]   D. d. Ridder, O. Kouropteva, O. Okun, M. Pietikainen, and R. P. W. Duin, "Supervised locally linear embedding," presented at the Artificial Neural Networks and Neural Information Processing-ICANN/ICONIP 2003, 2003.
[11]   Z. Zhang, T. W. S. Chow, and M. Zhao, "M-Isomap: Orthogonal Constrained Marginal Isomap for Nonlinear Dimensionality Reduction," IEEE Transactions on Cybernetics  vol. 43, pp. 180 - 191, 2013.
[12]   R. Hettiarachchi and J.F.Peters, "Multi-manifold LLE learning in pattern recognition," Pattern Recognition, vol. 48, pp. 2947–2960, 2015.
[13]   F. Aeini, A. M. E. Moghadam, and F. MAhmoudi, "Supervised hierarchical neighborhood graph construction for manifold learning," Signal, Image and Video Processing, vol. 12, pp. 1-9, 2018.
[14]   C.-S. Lee, A. Elgammal, and M. Torki, "Learning representations from multiple manifolds," Pattern Recognition, vol. 50, pp. 74-87, 2016.
[15]   H.-T. Chen, H.-W. Chan, and T.-L. .Liu, "Local discriminant embedding and its variants," presented at the 2005 IEEE Computer Society Conferenceon Compute rVision and Pattern Recognition, CVPR, 2005.
[16]   M. Sugiyama, "Local fisher discriminant analysis for supervised dimensionality reduction," presented at the Proceedings of the 23rd International Conferenceon Machine Learning, Pittsburgh, Pennsylvania, 2006.
[17]   D. Cai, X. He, K. Zhou, J. Han, and H. Bao, "Locality sensitive discriminantan alysis," IJCAI, pp. 708–713, 2007.
[18]   X. He, M. Ji, and H. Bao, "Graph embedding with constraints," IJCAI, vol. 9, pp. 1065–1070, 2009.
[19]   Y. Bengio, J. F. Paiement, P. Vincent, O. Delalleau, N. L. Roux, and M. Ouimet, "Out-of-sample extensions for LLE, Isomap, MDS, eigenmaps, and spectral clustering," presented at the Proceedings of the Neural Information Processing Systems, MIT Press, Cambridge, MA, 2004.
[20]   W. K. Wonga and H. T. Zhao, "Supervised optimal locality preserving projection," Pattern Recognition, vol. 45 pp. 186–197, 2012.
[21]   B. Yang, M. Xiang, and Y. Zhang, "Multi-manifold discriminant Isomap for visualization and classification," Pattern Recognition, vol. 55, pp. 215–230, 2016.
[22]   N. Gu, M. Fan, H. Qiao, and B. Zhang, "Discriminative Sparsity Preserving Projections for Semi-Supervised Dimensionality Reduction," IEEE SIGNAL PROCESSING LETTERS, vol. 19, pp. 391-394, 2012.
[23]   M. Fan, X. Zhang, Z. Lin, Z. Zhang, and H. Bao, "A Regularized Approach for Geodesic-Based Semisupervised Multimanifold Learning," IEEE TRANSACTIONS ON IMAGE PROCESSING, vol. 23, pp. 2133-2147, 2014.
[24]   B. Fischer and J. M. Buhmann, "Path-based clustering for grouping of smooth curves and texture segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, pp. 513–518, 2003.
[25]   T. Evgeniou, T. Poggio, M. Pontil, and A. Verric, "Regularization and statistical learning theory for data analysis," Computational Statistics & Data Analysis, vol. 38, pp. 421-432, 2002.
[26]   M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples," J. Mach. Learn. Res., vol. 7, pp. 2399-2434, 2006.
[27]   N. Gu, M. Fan, H. Qiao, and B. Zhang, "Discriminative Sparsity Preserving Projections for Semi-Supervised Dimensionality Reduction," IEEE Signal Processing Letters, vol. 19, pp. 391 - 394, 2012.
[28]   L. v. d. Maaten, E. Postma, and J. v. d. Herik, "Dimensionality Reduction: A Comparative Review," Tilburg University Technical Report, TiCC-TR 2009-
[29]   J. Lu, Y.-P. Tan, and G. Wang, "Discriminative Multimanifold Analysis for Face Recognition from a Single Training Sample per Person," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, pp. 39 - 51, 2013.
[30]   M. Fornasier and F. Pitolli, "Adaptive iterative thresholding algorithms for magnetoence phalography (MEG)," Comput.Appl.Math., vol. 211, pp. 386–395, 2008.
[31]   F. Nie, H. Huang, X. Cai, and C. Ding, "Efficient and robust feature selection via joint ℓ2,1-norms minimization," Advances in Neural Information Processing Systems, 2010.
[32]   C. EduardoThomaz and G. AntonioGiraldi, "A new ranking method for Principal Components Analysis and its application to face image analysis," Image and Vision Computing, vol. 28, pp. 902-913, 2010.
[33]             T. C. E. and G. G. A., "A new ranking method for Principal Components Analysis and its application to face image analysis," Image and Vision Computing, vol. 28, pp. 902-913, 2010.
Journal of Soft Computing and Information Technology
Volume 8, Issue 4 - Serial Number 26
Winter
February 2020
Pages 1-16

Files

Share

How to cite

Statistics