TSGET: Two-Stage Global Enhanced Transformer for Automatic Radiology Report Generation
Xiulong Yi, You Fu, Ruiqing Liu +2 more
Recently, automatic radiology report generation, which targets to generate multiple sentences that can accurately describe medical observations for given X-ray images, has gained increasing attention. Existing methods commonly employ the attention mechanism for accurate word generation. However, such attention-based methods fail to leverage useful image-level global features, thereby limiting the model's reasoning ability. To tackle this challenge, we propose two-stage global enhancement layers to facilitate the Transformer to generate more reliable reports from a global perspective. Specifically, the 1st Global Enhancement Layer (1st GEL) is designed to capture the global visual context features by establishing the relationships between image-level global features and previously generated words. The 2nd Global Enhancement Layer (2nd GEL) is devised to capture the region-global level…