COT2DNet: Contextual 2D Attention for Scene Text Recognition

Abstract

Abstract Although scene text recognition (STR) methods have made great progress, reading the text of irregularly shaped scenes remains a challenge. The current conversion of two-dimensional (2D) images to one-dimensional (1D) feature maps for sequence recognition may lose a large amount of spatial information and cannot recognize text with a large degree of curvature. In this paper, we propose an network infrastructure to recognize texts of arbitrary shapes, named Contextual 2D Attention for Scene Text Recognition Network (COT2DNet). Firstly, deformable convolution is added to the feature extraction module, which can perform feature extraction according to the shape of the character and expand the receptive field of feature extraction, in the Transformer encoding stage, the multi-head attention is changed to Contextual 2D Multi-head Attention (COT2D), which can makes full use of contextual information between Q,K,V neighbouring keys to facilitate attentional learning, which improves the model's ability to perceive contextual information in text image recognition tasks, and at the same time can effectively learn complex background image feature representations. Due to the full-image propagation property, it may be inefficient to capture short-term dependencies using multi-stacked attentional blocks, and Multiscale position feedforward (CFEED) is used to acquire short-term dependencies for 2D images. Thus, the model can capture features of text images at multiple scales. Experiments show that, together with a visual model-based recogniser, COT2DNet significantly improves lexical contextual links, achieving state-of-the-art performance in public benchmarking.