Computer Science · Computer Vision and Pattern Recognition
Research on Image Tibetan Caption Generation Method Fusion Attention Mechanism
Jianjun Xia, Xin Yang, Qiong Ni, Dingguo Gao
See on the mapAbstract
Image caption generation combines computer vision and natural language processing to generate natural language descriptions of images. With the rapid development of deep learning, the performance of end-to-end image caption generation research based on Chinese and English datasets has been improved. However, there is a lack of Tibetan caption data samples for low-resource languages, and the research on image Tibetan caption generation is almost blank. Therefore, in order to promote the research and development of multilingual image generation and improve the accuracy of Tibetan image caption, VGG19, InceptionV3 and ResNet101 are used as the backbone network, and group convolution is used to replace the fully connected operation of the attention mechanism. A CNN-based An image Tibetan caption generation method that integrates with the LSTM encoding and decoding network to improve the attention mechanism. Experiments on the Flickr8K and Flickr30K-tic Tibetan caption datasets show that this method is superior to the traditional attention mechanism image caption generation method, and has achieved better performance improvement in BLEU and ROUGE-L image caption evaluation indicators.