arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Computer Networks and Communications

TCP-QNCC: congestion control algorithm based on deep Q-network

Xiang Huang, Zhu Hua, Yang Li, Chun-lin Yin, Jie Li, Hang Li

See on the map

Abstract

When the demand for network resources exceeds the available capacity of the network, network congestion occurs on the Internet. Network congestion can cause transmission delays, packet loss, and even congestion collapse. When this happens, all communication across the network comes to a halt. Therefore, congestion has long been a concern. Although traditional congestion control algorithms have performed well in early simple networks, their limitations prevent them from being effective before network congestion occurs, making them difficult to cope with the challenges of network complexity and service diversification. In this paper, we model network congestion control as a Markov decision process and optimize congestion control strategies using a deep Q network in deep reinforcement learning, proposing a congestion control algorithm, QNCC, that is purely data-driven and does not rely on any assumptions. QNCC uses a fully connected neural network to approximate the value function, enabling it to automatically learn features and have good generalization ability. Experiments show that QNCC performs better overall than traditional congestion control algorithms such as TCP-Cubic and TCP-Vegas in multiple network scenarios.

More like this