COM-transformers: research on automatic knowledge graph construction based on transformer

Abstract

We have proposed the first comprehensive scheme for automatic knowledge base construction, which targets a widely used commonsense knowledge graphs: ATOMIC. Unlike many traditional knowledge bases that store structured information using standardized templates, commonsense knowledge bases only store unstructured text descriptions. We believe that the development of generative commonsense models is an essential step towards the automatic construction of commonsense knowledge. To this end, we have introduced a novel method called COM-Transformers, which learns to generate diverse natural language descriptions of commonsense knowledge. Although commonsense modeling is challenging, our investigation shows that by turning implicit knowledge in pre-trained language models into explicit knowledge, we have achieved promising results on ATOMIC. The results demonstrate that COM-Transformers is able to generate novel knowledge of high quality that is evaluated favorably by humans, comprehensively improving performance on the benchmark. Our research results indicate that automatic generation of commonsense knowledge bases using commonsense generative models may quickly become a viable alternative to extraction approaches.