Journal of Soft Computing and Information Technology

Journal of Soft Computing and Information Technology

A Benchmark for Analyzing Knowledge Graph Embedding for Link Prediction Problem in Low-Resource Languages

Document Type : Persian Original Article

Authors
Najmeh Torabian 1
Behrooz Minaei-Bidgoli 2
Mohsen Jahanshahi 1
1 Department of Computer Engineering, Central Tehran Branch, Islamic Azad University, Tehran, Iran.
2 Department of Computer Engineering, Iran University of Science and Technology, Tehran, Iran.
Abstract
Link prediction in knowledge graphs addresses predicting missing entities or relations of a knowledge graph, typically using knowledge graph embedding techniques. Training these models on low-resource-language knowledge graphs presents unique challenges, which have not been thoroughly addressed in the literature, and correspondingly there is no benchmark for evaluating link prediction methods on such graphs. These knowledge graphs often have unique topologies due to the characteristics of low-resource languages. Many knowledge graphs are derived from encyclopedias like Wikipedia, which in low-resource languages may have many propositions from common subjects and/or facts and few from less common ones, leading to distinctive topologies in the extracted knowledge graph. FarsBase, as the knowledge graph related to the Persian language, exemplifies these properties. Originating from Persian Wikipedia, it has some relation-types with many numbers of instances and some other relation types with very few instances. This paper introduces "FarsPishBin," a lightly-pruned version of FarsBase, as a benchmark for low-resource-language knowledge graph embedding. The authors argue that translational models are likely to outperform other embedding models on this benchmark. To check the mentioned hypothesis, the popular embedding models are evaluated on FarsPishBin and the experiments prove that translational models (as expected) perform best. This benchmark aims to serve as a standard platform for future-coming models addressing link prediction in low-resource-language knowledge graphs.
 
Keywords
Link prediction
Knowledge graph embedding
Link prediction benchmark
[1] H. Yu, H. Li, D. Mao, and Q. Cai, “A domain knowledge graph construction method based on Wikipedia,” J. Inf. Sci., vol. 47, no. 6, pp. 783–793, 2021
[2] M. Yahya, J. G. Breslin, and M. I. Ali, “Semantic web and knowledge graphs for industry 4.0,” Appl. Sci., vol. 11, no. 11, pp. 5110, 2021.
[3] A. Bordes, N. Usunier, A. Garcia-Durán, J. Weston, and O. Yakhnenko, “Translating embeddings for modeling multi-relational data,” in Advances in Neural Information Processing Systems, Neural information processing systems foundation, 2013.
[4] M. Shamsfard and Y. Ghazanfari, “Augmenting farsnet with new relations and structures for verbs,” in Proceedings of the 8th Global WordNet Conference, GWC 2016, 2016, pp. 377–383.
[5] M. Asgari-Bidhendi, A. Hadian, and B. Minaei-Bidgoli, “FarsBase: The Persian knowledge graph,” Semant. Web, vol. 10, no. 6, pp. 1169–1196, 2019.
[6] S. Auer, C. Bizer, G. Kobilarov, J. Lehmann, R. Cyganiak, and Z. Ives, “DBpedia: A Nucleus for a Web of Open Data,” in The Semantic Web, K. Aberer, K.-S. Choi, N. Noy, D. Allemang, K.-I. Lee, L. Nixon, J. Golbeck, P. Mika, D. Maynard, R. Mizoguchi, G. Schreiber, and P. Cudré-Mauroux, Eds., Berlin, Heidelberg: Springer Berlin Heidelberg, pp. 722–735, 2007.
[7] K. Bollacker, C. Evans, P. Paritosh, T. Sturge, and J. Taylor, “Freebase: a collaboratively created graph database for structuring human knowledge,” in In SIGMOD Conference, pp. 1247–1250, 2008.
[8] R. Navigli and S. P. Ponzetto, “BabelNet: The automatic construction, evaluation and application of a wide-coverage multilingual semantic network,” Artif. Intell., vol. 193, pp. 217–250, 2012.
[9] D. Vrandečić and M. Krötzsch, “Wikidata: A free collaborative knowledgebase,” Commun. ACM, vol. 57, no. 10, pp. 78–85, 2014.
[10] T. Pellissier Tanon, G. Weikum, and F. Suchanek, “YAGO 4: A Reason-able Knowledge Base,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , pp. 583–596, 2020.
[11] A. Uyar and F. M. Aliyu, “Evaluating search features of Google Knowledge Graph and Bing Satori: Entity types, list searches and query interfaces,” Online Inf. Rev., vol. 39, no. 2, pp. 197–213, 2015.
[12] Z. Wang et al., “XLore: A large-scale English-Chinese bilingual knowledge graph,” in CEUR Workshop Proceedings, pp. 121–124, 2013.
[13] E. Montiel-Ponsoda and V. Rodr’iguez-Doncel, “Lynx: Building the legal knowledge graph for smart compliance services in multilingual Europe,” in Proceedings of the 1st workshop on LREC (language resources and technologies for the legal knowledge graph) workshop, pp. 19–22, 2018.
[14] J.-P. Prost, “Holinet: Holistic knowledge graph for french,” Journées Jointes des Group. Rech. Linguist. Informatique, Formelle Terrain, vol. 1, pp. 123–130, 2022.
[15] S. Y. Yoo and O. R. Jeong, “Automating the expansion of a knowledge graph,” Expert Syst. Appl., vol. 141, p. 112965, 2020.
[16] M. Mirzapour et al., “Introducing RezoJDM16k: a French KnowledgeGraph DataSet for link prediction,” in Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 5163–5169, 2022.
[17] H. Alberts et al., “VisualSem: A High-quality Knowledge Graph for Vision & Language,” in MRL 2021 - 1st Workshop on Multilingual Representation Learning, Proceedings of the Conference, pp. 138–152, 2021.
[18] Y. Lin, Z. Liu, M. Sun, Y. Liu, and X. Zhu, “Learning entity and relation embeddings for knowledge graph completion,” in Proceedings of the National Conference on Artificial Intelligence, AI Access Foundation, pp. 2181–2187, 2015.
[19] Z. Wang, J. Zhang, J. Feng, and Z. Chen, “Knowledge graph embedding by translating on hyperplanes,” in Proceedings of the National Conference on Artificial Intelligence, AI Access Foundation, pp. 1112–1119, 2014.
[20] Z. Sun, Z. H. Deng, J. Y. Nie, and J. Tang, “Rotate: Knowledge graph embedding by relational rotation in complex space,” in 7th International Conference on Learning Representations, ICLR 2019, pp. 1-18, 2019.
[21] M. Nickel, V. Tresp, and H. P. Kriegel, “A three-way model for collective learning on multi-relational data,” in Proceedings of the 28th International Conference on Machine Learning, ICML 2011, pp. 809–816, 2011.
[22] M. Nickel, L. Rosasco, and T. Poggio, “Holographic embeddings of knowledge graphs,” in 30th AAAI Conference on Artificial Intelligence, AAAI 2016, pp. 1955–1961, 2016.
[23] T. Trouillon, J. Welbl, S. Riedel, E. Ciaussier, and G. Bouchard, “Complex embeddings for simple link prediction,” in 33rd International Conference on Machine Learning, ICML 2016, pp. 3021–3032, 2016.
[24] B. Shi and T. Weninger, “ProjE: Embedding projection for knowledge graph completion,” in 31st AAAI Conference on Artificial Intelligence, AAAI 2017, pp. 1236–1242, 2017.
[25] D. Q. Nguyen, D. Q. Nguyen, T. D. Nguyen, and D. Phung, “A convolutional neural network-based model for knowledge base completion and its application to search personalization,” Semant. Web, vol. 10, no. 5, pp. 947–960, 2019.
[26] T. Dettmers, P. Minervini, P. Stenetorp, and S. Riedel, “Convolutional 2D knowledge graph embeddings,” in 32nd AAAI Conference on Artificial Intelligence, AAAI 2018, pp. 1811–1818, 2018.
[27] C. Shang, Y. Tang, J. Huang, J. Bi, X. He, and B. Zhou, “End-to-end structure-aware convolutional networks for knowledge base completion,” in 33rd AAAI Conference on Artificial Intelligence, AAAI 2019, 31st Innovative Applications of Artificial Intelligence Conference, IAAI 2019 and the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2019, pp. 3060–3067, 2019.
[28] B. Yang, W. tau Yih, X. He, J. Gao, and L. Deng, “Embedding entities and relations for learning and inference in knowledge bases,” in 3rd International Conference on Learning Representations, ICLR 2015 Conference Track Proceedings, pp. 1-12, 2015.
[29] X. Han et al., “OpenKE: An open toolkit for knowledge embedding,” in EMNLP 2018 - Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Proceedings, pp. 139–144, 2018.
[30] S. Broscheit, K. Gashteovski, Y. Wang, and R. Gemulla, “Can we predict new facts with open knowledge graph embeddings? A benchmark for open link prediction,” in Proceedings of the Annual Meeting of the Association for Computational Linguistics, pp. 2296–2308, 2020.
[31] C. Fellbaum, “WordNet,” in Theory and Applications of Ontology: Computer Applications, pp. 231–243, 2010.
[32] M. Wang, L. Qiu, and X. Wang, “A survey on knowledge graph embeddings for link prediction,” Symmetry (Basel)., vol. 13, no. 3, 2021.
[33] P. Rosso, D. Yang, and P. Cudré-Mauroux, “Beyond Triplets: Hyper-Relational Knowledge Graph Embedding for Link Prediction,” in The Web Conference 2020 - Proceedings of the World Wide Web Conference, WWW 2020, pp. 1885–1896, 2020.
Journal of Soft Computing and Information Technology
Volume 13, Issue 4
Winter - March
Winter 2025
Pages 38-48