arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Artificial Intelligence

Deep Learning-Based Multi-Modal Semantic Communication with Automated Error Detection and Correction

Haseeb Ur Rehman, Kashif Bashir, Muhammad Afzal Shah, Muhammad Suleman Soomro, Rumaisa Rafiq, Shayan Ul Abidin

· European Journal of Applied Science Engineering and Technology

See on the map

Abstract

Semantic communication is a potential promising paradigm in next-generation communication systems, in which the emphasis is laid on the transfer of meaningful information, as opposed to bare data. Although it has the potential to enhance efficiency of communication and exchange of information intelligently, semantic distortion due to noise channels is a significant threat, especially in multi-modal communication situations. This paper presents a multi-modal semantic communication system based on deep learning that features fully automated error detection and correction. The framework shares visual and textual information with distinct encoders of image and text and combines their semantic representations in a common latent space and sends the combined representation across a noisy communication channel. The semantic representation sent is decoded to recreate the image at the receiver side and re-extract the equivalent textual information. To enhance resilience, a loss-based semantic error detection system is suggested to detect damaged transmissions and an automated correction plan is implemented each time the detected error is more than a pre-determined threshold. The suggested framework is tested on several benchmark data sets, such as MNIST, FashionMNIST, CIFAR-10, and EMNIST, to test its ability to withstand the needs of various data distributions. The experimental findings show that the proposed model can satisfy stable multi-modal semantic recovery and effective semantic error detection in all the tested datasets, and the correction mechanism offers some, but not significant, quantitative advantages in noisy environments. The framework proposed provides a dynamic and practical solution to intelligent multi-modal semantic communication in future communication networks like beyond-5G and 6G networks.

More like this