On Application-Layer DDoS Attack Detection in High-Speed Encrypted Networks

Abstract

Application-layer denial-of-service attacks have become a serious threat to modern high-speed computer networks and systems. Unlike network-layer attacks, application-layer attacks can be performed by using legitimate requests from legitimately connected network machines which makes these attacks undetectable for signature-based intrusion detection systems. Moreover, the attacks may utilize protocols that encrypt the data of network connections in the application layer making it even harder to detect attacker’s activity without decrypting users network traffic and violating their privacy. In this paper, we present a method which allows us to timely detect various applicationlayer attacks against a computer network. We focus on detection of the attacks that utilize encrypted protocols by applying an anomaly-detection-based approach to statistics extracted from network packets. Since network traffic decryption can violate ethical norms and regulations on privacy, the detection method proposed analyzes network traffic without decryption. The method involves construction of a model of normal user behavior by analyzing conversations between a server and clients. The algorithm is self-adaptive and allows one to update the model every time when a new portion of network traffic data is available. Once the model has been built, it can be applied to detect various types of application-layer denial-of- service attacks. The proposed technique is evaluated with realistic end user network traffic generated in our virtual network environment. Evaluation results show that these attacks can be properly detected, while the number of false alarms remains very low.