arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Computer Vision and Pattern Recognition

Cross-Modal Retrieval via Contrastive Representation Learning of Images and Text Descriptions

Zhaoxuan Li, Na Tang

· International Journal of Pattern Recognition and Artificial Intelligence

See on the map

Abstract

Cross-modal retrieval aims to bridge the semantic gap between heterogeneous modalities — such as images and text — by learning a shared embedding space for semantically aligned representation. While recent models have achieved impressive performance using large-scale contrastive pretraining and multimodal transformers, several fundamental challenges remain unresolved. These include the lack of interpretable latent alignment, vulnerability to distribution shifts, and instability in semantic correspondence across tasks and domains. In this paper, we propose a novel contrastive representation learning framework designed to enhance both the robustness and interpretability of cross-modal retrieval. Our method incorporates a hierarchical dual-stream encoder that preserves modality-specific structures while enabling semantic interaction through a concept-aligned projection layer. The model is optimized via a contrastive loss with semantic-aware calibration, encouraging consistent feature correspondence across modalities. We provide a rigorous theoretical analysis of the latent projection space, and demonstrate through extensive experiments on MS-COCO, Flickr30K, and RSICD that our approach outperforms strong baselines not only in retrieval accuracy but also in robustness under noise and interpretability via semantic stability selection. The proposed framework is further validated through ablation studies that isolate the contributions of architectural components and training strategies. Our results confirm that semantic disentanglement and hierarchical encoding jointly improve retrieval quality, cross-domain generalization, and feature transparency. The framework offers a scalable and theoretically grounded solution for reliable and explainable multimodal retrieval.

More like this