Link Prediction Model for Automated Extraction of Hypernym and Hyponym Relations in Multiple Languages
Abstract
In natural language processing, hypernym–hyponym relations are the core of the body of knowledge and are useful for many downstream tasks, for example, technical trend analysis and patent examination. However, it is very costly to manually maintain these relations between terms. In this paper, we extract hypernym–hyponym relations from patent text data written in Japanese, English, and Chinese, and automatically construct a multilingual thesaurus. The proposed method consists of the following two steps. First, we use a generative adversarial network (GAN) to identify the terms in a hypernym–hyponym relation. Then, ConvE and GraphSAGE are combined to predict links on the graph of hypernym–hyponym relations constructed in the previous step, and to predict missing edges that should be in a hypernym–hyponym relation. In experiments conducted to demonstrate the effectiveness of the proposed method, it was found that our method outperformed previous methods in both the identification of hypernym–hyponym relations using GAN and link prediction using a combination of ConvE and GraphSAGE. We constructed a classifier that can discriminate between hypernym–hyponym relations using a trained Discriminator with GAN. In our experiments, we achieved a recall rate of 0.936 in English. And we propose a new method that can automatically complement missing hypernym–hyponym relations. The proposed model achieved a score of 97.3 on the H@10 evaluation index with respect to the prediction of Chinese hypernym–hyponym relations.
References
Hearst, M. A., “Automatic Acquisition of Hyponyms from Large Text Corpora,” in Proceedings of the 14th International Conference on Computational Linguistics, pp. 539–545, 1992.
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A. and Bengio, Y., “Generative Adversarial Nets,” Proceedings of the 27th International Conference on Neural Information Processing Systems, pp. 2672–2680, 2014.
Huang, Z., Li, Z., Jiang, H., Cao, T., Lu, H., Yin, B., Subbian, K., Sun, Y., and Wang, W., “Multilingual Knowledge Graph Completion with Self-Supervised Adaptive Graph Alignment,” Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pp. 474–485, 2022.
Chen, X., Chen, M., Fan, C., Uppunda, A., Sun, Y. and Zaniolo, C., “Multilingual Knowledge Graph Completion via Ensemble Knowledge Transfer,” Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 3227–3228, 2020.
Dettmers, D., Minervini, P., Stenetorp, P., and Riedel, S., “Convolutional 2D Knowledge Graph Embedding,” Proceedings of the 32th AAAI Conference on Ar-tificial Intelligence, 2018.
Hamilton, W.L., Ying, R. and Leskovec, J., “Inductive Representation Learning on Large Graphs,” Proceedings of the 31st International Conference on Neural Infor-mation Processing Systems, pp. 1025–1035, 2017.
Nanba, H., Mayumi, S., and Takezawa, T. “Automatic Construction of a Bilingual Thesaurus using Citation Analysis,” Proceedings of the 4th International CIKM Workshop on Patent Information Retrieval (PaIR’11), pp. 25–30, 2011.
Yu, C., Han, J., Zhang, H. and Ng, W., “Hypernymy Detection for Low-Resource Languages via Meta Learning,” Proceedings of the 58th Annual Meeting of the As-sociation for Computational Linguistics, pp. 3651–3656, 2020.
Upadhyay, S., Vyas, Y., Carpuat, M. and Roth, D., “Robust Cross-lingual Hypernymy Detection using Dependency Context,” Proceedings of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics, pp. 607–618, 2018.
Kober, T., Weeds, J., Bertolini, L. and Weir, D., “Data Augmentation for Hypernymy Detection,” Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pp. 1034–1048, 2021.
Chavdarova, T. and Fleuret, F., “SGAN: An Alternative Training of Generative Adversarial Networks,” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018, pp. 9407–9415, 2018.
Bordes, A., Usunier, N., García-Durán, A., Weston, J. and Yakhnenko, O., “Trans-lating Embeddings for Modeling Multi-relational Data,” Proceedings of the 17th Conference on Neural Information Processing Systems, pp. 2787–2795, 2013.
Yang, B., Yih, W., He, X., Gao, J, and Deng, L., “Embedding Entities and Relations for Learning and Inference in Knowledge Bases,” Proceedings of the 3rd Interna-tional Conference on Learning Representations, 2015.
Trouillon, T., Welbl, J., Riedel, S., Gaussier, É. and Bouchard, G., “Complex Em-beddings for Simple Link Prediction,” Proceedings of the 33rd International Con-ference on Machine Learning, pp. 2071–2080, 2016.
Kipf, T.N. and Welling, M., “Semi-supervised Classification with Graph Convolu-tional Networks,” arXiv preprint arXiv:1609.02907, 2016.
Schlichtkrull, M., Kipf, T. N., Bloem, P., Berg, R. v. d., Titov, I. and Welling, M., “Modeling Relational Data with Graph Convolutional Networks,” arXiv preprint arXiv:1703.06103, 2017.
Jiang, X., Wang, Q. and Wang, B., “Adaptive Convolution for Multi-relational Learning,” Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, Vol. 1, pp. 978–987, 2019.
Perozzi, B., Al-Rfou, R. and Skiena, S., “DeepWalk: Online Learning of Social Representations,” Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 701–710, 2014.
