Computer Science · Computer Networks and Communications
TCP-QNCC: congestion control algorithm based on deep Q-network
Xiang Huang, Zhu Hua, Yang Li, Chun-lin Yin, Jie Li, Hang Li
See on the mapAbstract
When the demand for network resources exceeds the available capacity of the network, network congestion occurs on the Internet. Network congestion can cause transmission delays, packet loss, and even congestion collapse. When this happens, all communication across the network comes to a halt. Therefore, congestion has long been a concern. Although traditional congestion control algorithms have performed well in early simple networks, their limitations prevent them from being effective before network congestion occurs, making them difficult to cope with the challenges of network complexity and service diversification. In this paper, we model network congestion control as a Markov decision process and optimize congestion control strategies using a deep Q network in deep reinforcement learning, proposing a congestion control algorithm, QNCC, that is purely data-driven and does not rely on any assumptions. QNCC uses a fully connected neural network to approximate the value function, enabling it to automatically learn features and have good generalization ability. Experiments show that QNCC performs better overall than traditional congestion control algorithms such as TCP-Cubic and TCP-Vegas in multiple network scenarios.