A Novel Fuzzy Learning Model based on Forgetting Factor

Document Type : Persian Original Article

Author

Faculty Member

Abstract

Biological observations, indicate that amnesia is an integral part of the human learning system. Thus, amnesia in learning algorithms is not necessarily destructive and can be constructive. In implementation, due to space constraints and the number of neurons, a limited number of training patterns can be taught to the network. Consequently, to be able to obtain long-term learning capability, it must possess a kind of forgetting mechanism to make space for new learning patterns. Thus, a type of forgetting mechanism similar to the function of the human brain is necessitated. The need for a forgetting mechanism is more acute in online training. Amnesia is modeled as the loss of information from memory. In this paper, the ALM, which is one of the most widely used methods, is employed. The selected algorithm models the system based on the distribution of ink drops based on training data. In this method, in all the implementations, the amplitude of the ink drops on the IDS planes remains unchanged, and no amnesia occurs, which is contrary to biological observations. In this work, the forgetting mechanism is added to the presented algorithm, and simulations in the modeling process are investigated.

Keywords

References

[1] Liu, F., et al., A New Fuzzy Spiking Neural Network Based on Neuronal Contribution Degree. IEEE Transactions on Fuzzy Systems, 2021.
[2] Qasem, S.N. and A. Mohammadzadeh, A deep learned type-2 fuzzy neural network: Singular value decomposition approach. Applied Soft Computing, 2021. 105: p. 107244.
[3] Hebb, D.O., The organization of behavior: A neuropsychological theory. 2005: Psychology Press.
[4] Fox, M.D. and M.E. Raichle, Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging. Nature reviews neuroscience, 2007. 8(9): p. 700-711.
[5] Abbott, L.F. and S.B. Nelson, Synaptic plasticity: taming the beast. Nature neuroscience, 2000. 3(11): p. 1178-1183.
[6] Pan, H.-j., et al. A Category Theory Model for Learning and Memory of the Human Brain. in 2010 International Conference on Digital Manufacturing & Automation. 2010. IEEE.
[7] French, R.M., Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 1999. 3(4): p. 128-135.
[8] Averkin, A. and S. Yarushev, Review of research in the field of developing methods to extract rules from artificial neural networks. Journal of Computer and Systems Sciences International, 2021. 60(6): p. 966-980.
[9] Nakayama, H. and K. Yoshii. Active forgetting in machine learning and its application to financial problems. in Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium. 2000. IEEE.
[10] Salganicoff, M., Explicit Forgetting Algorithms for Memory Based Learning. 1993.
[11] Panda, P., et al., Asp: Learning to forget with adaptive synaptic plasticity in spiking neural networks. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, 2017. 8(1): p. 51-64.
[12] Auge, D., et al., A survey of encoding techniques for signal processing in spiking neural networks. Neural Processing Letters, 2021. 53(6): p. 4693-4710.
[13] Kato, A. and K. Morita, Forgetting in reinforcement learning links sustained dopamine signals to motivation. PLoS computational biology, 2016. 12(10): p. e1005145.
[14] Shouraki, S.B. and N. Honda. Fuzzy controller design by an active learning method. in 31th symposium of intelligent control. Tokyo, Japan. 1998.
[15] Merrikh-Bayat, F., S.B. Shouraki, and A. Rohani, Memristor crossbar-based hardware implementation of the IDS method. IEEE Transactions on Fuzzy Systems, 2011. 19(6): p. 1083-1096.
[16] Afrakoti, I.E.P., S.B. Shouraki, and B. Haghighat, An optimal hardware implementation for active learning method based on memristor crossbar structures. IEEE Systems Journal, 2014. 8(4): p. 1190-1199.
[17] Shahdi, S.A. and S.B. Shouraki. Supervised active learning method as an intelligent linguistic controller and its hardware implementation. in 2nd IASTEAD International Conference on Artificial Intelligence and Applications (AIA'02), Malaga, Spain. 2002.
[18] Klidbary, S.H., et al. Outlier robust fuzzy active learning method (ALM). in 2017 7th international conference on computer and knowledge engineering (ICCKE). 2017. IEEE.
[19] Klidbary, S.H., S.B. Shouraki, and I.E.P. Afrakoti, An adaptive efficient memristive ink drop spread (IDS) computing system. Neural Computing and Applications, 2019. 31(11): p. 7733-7754.
[20] Jokar, E., et al., Hardware-Algorithm Co-Design of a Compressed Fuzzy Active Learning Method. IEEE Transactions on Circuits and Systems I: Regular Papers, 2020. 67(12): p. 4932-4945.
[21] Klidbary, S.H. and S.B. Shouraki, A novel adaptive learning algorithm for low-dimensional feature space using memristor-crossbar implementation and on-chip training. Applied Intelligence, 2018. 48(11): p. 4174-4191.
[22] Javadian, M., et al., Refining membership degrees obtained from fuzzy C-means by re-fuzzification. Iranian Journal of Fuzzy Systems, 2020. 17(4): p. 85-104.
[23] سجاد حق زاد کلیدبری، سعید باقری شورکی، ارائه اپراتور جدید جایگزین پخش قطره جوهر در روش یادگیری فعال. مجله مهندسی برق دانشگاه تبریز, 2019.
[24] Klidbary, S.H., S.B. Shouraki, and B. Linares-Barranco, Digital hardware realization of a novel adaptive ink drop spread operator and its application in modeling and classification and on-chip training. International Journal of Machine Learning and Cybernetics, 2019. 10(9): p. 2541-2561.
[25] Javadian, M., A. Hejazi, and S.H. Klidbary, Obtaining Fuzzy Membership Function of Clusters With the Memristor Hardware Implementation and On-Chip Learning. IEEE Transactions on Emerging Topics in Computational Intelligence, 2022.
[26] Murakami, M. and N. Honda, A study on the modeling ability of the IDS method: A soft computing technique using pattern-based information processing. International journal of approximate reasoning, 2007. 45(3): p. 470-487.
[27] Murakami, M. and N. Honda. Classification performance of the IDS method based on the two-spiral benchmark. in 2005 IEEE International Conference on Systems, Man and Cybernetics. 2005. IEEE.
[28] Hwang, J.-N., et al., Regression modeling in back-propagation and projection pursuit learning. IEEE Transactions on neural networks, 1994. 5(3): p. 342-353.
Journal of Soft Computing and Information Technology
Volume 11, Issue 2 - Serial Number 36
Summer
September 2022
Pages 53-63

Files

Share

How to cite

Statistics