The Experts below are selected from a list of 4569 Experts worldwide ranked by ideXlab platform
Randy H Katz - One of the best experts on this subject based on the ideXlab platform.
-
Fast and memory-efficient regular expression matching for deep Packet inspection
2006 Symposium on Architecture For Networking And Communications Systems, 2006Co-Authors: Fang Yu, Tirunellai V Lakshman, Zhifeng Chen, Yanlei Diao, Randy H KatzAbstract:Packet content scanning at high speed has become extremely important due to its applications in network security, network monitoring, HTTP load balancing, etc. In content scanning, the Packet Payload is compared against a set of patterns specified as regular expressions. In this paper, we first show that memory requirements using traditional methods are prohibitively high for many patterns used in Packet scanning applications. We then propose regular expression rewrite techniques that can effectively reduce memory usage. Further, we develop a grouping scheme that can strategically compile a set of regular expressions into several engines, resulting in remarkable improvement of regular expression matching speed without much increase in memory usage. We implement a new DFA-based Packet scanner using the above techniques. Our experimental results using real-world traffic and patterns show that our implementation achieves a factor of 12 to 42 performance improvement over a commonly used DFA- based scanner. Compared to the state-of-art NFA-based implementation, our DFA-based Packet scanner achieves 50 to 700 times speedup.
-
high speed deep Packet inspection with hardware support
High speed deep packet inspection with hardware support, 2006Co-Authors: Randy H KatzAbstract:In this dissertation, we developed high speed Packet processing algorithms for new services such as network intrusion detection, high speed firewalls, Network Address Translation (NAT), Hypertext Transfer Protocol (HTTP) load balancing, Extensible Markup Language (XML) processing, and Transmission Control Protocol (TCP) offloading. These new services have stringent requirements for speed, extensibility, scalability, and cost-effectiveness. For example, some services require rapid scanning of Packets against thousands of known patterns. Traditional Packet handling techniques, such as next hop forwarding, focus on Packet headers only and fail to support these demanding requirements. This thesis research aims to provide fast and efficient deep Packet inspection techniques that can function on the entire Packet content rather than just the header. To keep up with high speed Packet processing in existing networks, we proposed deep Packet inspection schemes that are optimized for new technologies such as Ternary Content Addressable Memory (TCAM) and multi-core processors. We propose algorithms that work both on Packet headers and Packet Payload. Our techniques form a cohesive architecture that can perform Gigbit rate Packet scanning against thousands of sophisticated patterns.
-
gigabit rate Packet pattern matching using tcam
International Conference on Network Protocols, 2004Co-Authors: Fang Yu, Randy H Katz, T V LakshmanAbstract:In today's Internet, worms and viruses cause service disruptions with enormous economic impact. Current attack prevention mechanisms rely on end-user cooperation to install new system patches or upgrade security software, yielding slow reaction time. However, malicious attacks spread much faster than users can respond, making effective attack prevention difficult network-based mechanisms, by avoiding end-user coordination, can respond rapidly to new attacks. Such mechanisms require the network to inspect the Packet Payload at line rates to detect and filter those Packets containing worm signatures. These signature sets are large (e.g., thousands) and complex. Software-only implementations are unlikely to meet the performance goals. Therefore, making a network-based scheme practical requires efficient algorithms suitable for hardware implementations. This work develops a ternary content addressable memory (TCAM) based multiple-pattern matching scheme. The scheme can handle complex patterns; such as arbitrarily long patterns, correlated patterns, and patterns with negation. For the ClamAv virus database with 1768 patterns whose sizes vary from 6 bytes to 2189 bytes, the proposed scheme can operate at a 2 Gbps rate with a 240 KB TCAM.
Xucheng Song - One of the best experts on this subject based on the ideXlab platform.
-
deep anomaly detection in Packet Payload
Neurocomputing, 2021Co-Authors: Jiaxin Liu, Xucheng Song, Yingjie Zhou, Xi Peng, Yanru Zhang, Pei Liu, Ce ZhuAbstract:Abstract With the wide deployment of edge devices, a variety of emerging applications have been deployed at the edge of network. To guarantee the safe and efficient operations of the edge applications, especially the extensive web applications, it is important and challenging to detect Packet Payload anomalies, which can be expressed as a number of specific strings that may cause attacks. Although some approaches have achieved remarkable progress, they are with limited applications since these approaches are dependent on in-depth expert knowledge, e.g., signatures describing anomalies or communication protocol at the application level. Moreover, they might fail to detect the Payload anomalies that may have long-term dependency relationships at the edge of network. To overcome these limitations and adaptively detect anomalies from Packet Payloads, we propose a deep learning based framework which does not rely on any in-depth expert knowledge and is capable of detecting anomalies that have long-term dependency relationships. The proposed framework consists of two parts. First, a novel block sequence construction method is proposed to obtain a valid expression of a Payload. The block sequence could encapsulate both the high-dimension information and the underlying sequential information which facilitate the anomaly detection. Secondly, we design a detection model to learn two different dependency relationships within the block sequence, which is based on Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN) and Multi-head Self Attention Mechanism. Furthermore, we cast the anomaly detection as a classification problem and employ a classifier with attention mechanism to integrate information and detect anomalies. Extensive experimental results on three public datasets indicate that our model could achieve a higher detection rate, while keeping a lower false positive rate compared with two traditional machine learning methods and three state-of-the-art methods.
-
deep anomaly detection in Packet Payload
arXiv: Signal Processing, 2019Co-Authors: Xucheng Song, Yingjie Zhou, Xi Peng, Yanru Zhang, Dapeng Oliver WuAbstract:With the widespread adoption of cloud services, especially the extensive deployment of plenty of Web applications, it is important and challenging to detect anomalies from the Packet Payload. For example, the anomalies in the Packet Payload can be expressed as a number of specific strings which may cause attacks. Although some approaches have achieved remarkable progress, they are with limited applications since they are dependent on in-depth expert knowledge, e.g., signatures describing anomalies or communication protocol at the application level. Moreover, they might fail to detect the Payload anomalies that have long-term dependency relationships. To overcome these limitations and adaptively detect anomalies from the Packet Payload, we propose a deep learning based framework which consists of two steps. First, a novel feature engineering method is proposed to obtain the block-based features via block sequence extraction and block embedding. The block-based features could encapsulate both the high-dimension information and the underlying sequential information which facilitate the anomaly detection. Second, a neural network is designed to learn the representation of Packet Payload based on Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN). Furthermore, we cast the anomaly detection as a classification problem and stack a Multi-Layer Perception (MLP) on the above representation learning network to detect anomalies. Extensive experimental results on three public datasets indicate that our model could achieve a higher detection rate, while keeping a lower false positive rate compared with five state-of-the-art methods.
David Hay - One of the best experts on this subject based on the ideXlab platform.
-
Leveraging traffic repetitions for high-speed deep Packet inspection
Proceedings - IEEE INFOCOM, 2015Co-Authors: Anat Bremler-barr, Shimrit Tzur David, YOTAM HARCHOL, David HayAbstract:Deep Packet Inspection (DPI) plays a major role in contemporary networks. Specifically, in datacenters of content providers, the scanned data may be highly repetitive. Most DPI engines are based on identifying signatures in the Packet Payload. This pattern matching process is expensive both in memory and CPU resources, and thus, often becomes the bottleneck of the entire application. In this paper we show how DPI can be accelerated by leveraging repetitions in the inspected traffic. Our new mechanism makes use of these repetitions to allow the repeated data to be skipped rather than scanned again. The mechanism consists of a slow path, in which frequently repeated strings are identified and stored in a dictionary, along with some succinct information for accelerating the DPI process, and a data path, where the traffic is scanned byte by byte but strings from the dictionary, if encountered, are skipped. Upon skipping, the data path recovers to the state it would have been in had the scanning continued byte by byte. Our solution achieves a significant performance boost, especially when data is from the same content source (e.g., the same website). Our experiments show that for such cases, our solution achieves a throughput gain of 1.25-2.5 times the original throughput, when implemented in software.
Ce Zhu - One of the best experts on this subject based on the ideXlab platform.
-
deep anomaly detection in Packet Payload
Neurocomputing, 2021Co-Authors: Jiaxin Liu, Xucheng Song, Yingjie Zhou, Xi Peng, Yanru Zhang, Pei Liu, Ce ZhuAbstract:Abstract With the wide deployment of edge devices, a variety of emerging applications have been deployed at the edge of network. To guarantee the safe and efficient operations of the edge applications, especially the extensive web applications, it is important and challenging to detect Packet Payload anomalies, which can be expressed as a number of specific strings that may cause attacks. Although some approaches have achieved remarkable progress, they are with limited applications since these approaches are dependent on in-depth expert knowledge, e.g., signatures describing anomalies or communication protocol at the application level. Moreover, they might fail to detect the Payload anomalies that may have long-term dependency relationships at the edge of network. To overcome these limitations and adaptively detect anomalies from Packet Payloads, we propose a deep learning based framework which does not rely on any in-depth expert knowledge and is capable of detecting anomalies that have long-term dependency relationships. The proposed framework consists of two parts. First, a novel block sequence construction method is proposed to obtain a valid expression of a Payload. The block sequence could encapsulate both the high-dimension information and the underlying sequential information which facilitate the anomaly detection. Secondly, we design a detection model to learn two different dependency relationships within the block sequence, which is based on Long Short-Term Memory (LSTM), Convolutional Neural Networks (CNN) and Multi-head Self Attention Mechanism. Furthermore, we cast the anomaly detection as a classification problem and employ a classifier with attention mechanism to integrate information and detect anomalies. Extensive experimental results on three public datasets indicate that our model could achieve a higher detection rate, while keeping a lower false positive rate compared with two traditional machine learning methods and three state-of-the-art methods.
Dapeng Oliver Wu - One of the best experts on this subject based on the ideXlab platform.
-
deep anomaly detection in Packet Payload
arXiv: Signal Processing, 2019Co-Authors: Xucheng Song, Yingjie Zhou, Xi Peng, Yanru Zhang, Dapeng Oliver WuAbstract:With the widespread adoption of cloud services, especially the extensive deployment of plenty of Web applications, it is important and challenging to detect anomalies from the Packet Payload. For example, the anomalies in the Packet Payload can be expressed as a number of specific strings which may cause attacks. Although some approaches have achieved remarkable progress, they are with limited applications since they are dependent on in-depth expert knowledge, e.g., signatures describing anomalies or communication protocol at the application level. Moreover, they might fail to detect the Payload anomalies that have long-term dependency relationships. To overcome these limitations and adaptively detect anomalies from the Packet Payload, we propose a deep learning based framework which consists of two steps. First, a novel feature engineering method is proposed to obtain the block-based features via block sequence extraction and block embedding. The block-based features could encapsulate both the high-dimension information and the underlying sequential information which facilitate the anomaly detection. Second, a neural network is designed to learn the representation of Packet Payload based on Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN). Furthermore, we cast the anomaly detection as a classification problem and stack a Multi-Layer Perception (MLP) on the above representation learning network to detect anomalies. Extensive experimental results on three public datasets indicate that our model could achieve a higher detection rate, while keeping a lower false positive rate compared with five state-of-the-art methods.
