Computer Science · Computer Vision and Pattern Recognition
Study on Image Caption Generation Model with Improved Attention Mechanism
Jinjin Wang, Wenhui Li, Shangyou Zeng
See on the mapAbstract
The image caption generation task uses computer vision technology to automatically complete the image target recognition, and then uses the machine translation model to express the content of the image into a natural language, which combines the research directions of the two fields of computer vision and natural language processing. In recent years, the image caption generation model is based on Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN), but this model suffers from inaccurate visual feature selection and slow training speed. In response to this problem, this paper proposes an image caption generation model based on improved attention mechanism and long short-term memory (LSTM) network. The model uses Vgg19 and Resnet101 in the convolutional neural network as feature extraction networks, and introduces improved the attention mechanism improves the evaluation index of image caption generation compared with the previous attention model. This article uses the public data sets Flickr8K and Flickr8k_cn (Flickr8K Chinese data set marked by native Chinese speakers) for training, and uses a variety of evaluation indicators (BLEU, ROUGE_L, CIDEr, METEOR) to verify the model. The experimental results show that, compared with the traditional attention mechanism, the improved attention mechanism image caption generation model in this paper has a significant improvement in accuracy, and it is better than the traditional model in the four evaluation indicators.