Knowledge Graph Representation Learning by Text Encoding and Graph Structure

Abstract

Knowledge graph representation learning aims to embed entities and relationships into low-dimensional space through knowledge graph embedding methods. Because knowledge graphs are incomplete, it is often necessary to complete the knowledge graph through representation learning methods. With the development of pre-trained language models, more and more research applies them to the field of knowledge graph representation learning, using the powerful semantic representation capabilities of pre-trained language models to improve the performance of knowledge graph embedding. Most of the existing methods make use of the semantic information of the triple text but do not fully consider the structural information of the triple and the graph structure information of the knowledge graph. The triple structure reflects the semantic information and relationship pattern of the triple, and the graph structure reflects the surrounding entity's semantic features. To address the above issues, this paper proposes a knowledge graph representation learning method named PREGSE, which is based on pre-trained language models and integrates graph structure information. Firstly, pre-trained language models are employed to encode triplets through text encoding, obtaining vectors for the triplets. Secondly, a graph attention network is utilized to learn various local graph structure information. Lastly, a multi-task learning strategy is applied to simultaneously learn triplet structure information and semantic information. We trained our model on the FB15k-237 and WN18RR datasets, and the results show that on the FB15k-237 dataset, our model improved the MRR metric by 27% and the Hits@10 metric by 8% compared to the StAR model. The experiments show that our model can further improve the performance of knowledge graph representation
learning.