arXiv.itsnick.co

an independent field guide to 23 million papers from arXiv & OpenAlex — not affiliated with arXiv.org

Computer Science · Computer Networks and Communications

A cloud/edge computing streaming system for network traffic monitoring and threat detection

William G. Hatcher, Wei Yu, James Nguyen, Sixiao Wei, Zhijiang Chen

· International Journal of Security and Networks

See on the map

Abstract

The unyielding trend of increasing cyber threats has made cyber security paramount in protecting personal and private intellectual property. To provide a highly secured network environment, network threat detection systems must handle real-time big data from varied places in enterprise networks. In this paper, we introduce a streaming-based threat detection system that can rapidly analyse highly intensive network traffic data in real-time, utilising streaming-based clustering algorithms to detect abnormal network activities. The developed system integrates the high-performance data analysis capabilities of Flume, Spark and Hadoop into a cloud-computing environment to provide network monitoring and intrusion detection. Our performance evaluation validates that the developed system can cope with a significant volume of streaming data in a high detection accuracy and good system performance. We further extend our system for edge computing and discuss some key challenges, as well as some potential solutions, aiming to improve the scalability of our system.

More like this