The Experts below are selected from a list of 75 Experts worldwide ranked by ideXlab platform
Johan Mazel - One of the best experts on this subject based on the ideXlab platform.
-
Sub-Space Clustering and Evidence Accumulation for Unsupervised Anomaly Detection in IP Networks
2016Co-Authors: Pedro Casas, Johan Mazel, Philippe Owezarski, Yann LabitAbstract:Network traffic Anomaly detection and analysis has been a hot research topic for many years. Current detection systems employ two different approaches to tackle the problem, even using signature-based detection methods or supervised machine-learning techniques. However, both approaches present serious ground limitations. The former fails to detect new unknown anomalies, the latter highly relies on labeled data for training, which is difficult and expensive to produce. These limitations become highly restrictive in current Internet traffic scenario, characterized by emerging network applications and new variants of network attacks. In this paper, we introduce a novel approach to detect network traffic attacks in a completely unsupervised fashion. The proposed method does not assume any Anomaly signature or particular model for Anomaly-free traffic, which allows for detection of previously unseen attacks. By combining the multiple evidence of traffic structure provided by sub-space clustering techniques, we show that our method can efficiently isolate and extract unknown anomalies buried inside large amounts of traffic. Apart from discovering new anomalies, the method automatically generates a new and easy-to-interpret signature for the novel Detected Anomaly, easing network administrator tasks. This new unsupervised Anomaly detection method is a powerful means to detect zero-day attacks in a changing environment, where signature-based or supervised learning may fail. We evaluate the ability of our promising proposal to discover a distributed attack in real traffic from the public MAWI traffic repository, discussing future directions and ongoing work.
-
Unsupervised network Anomaly detection
2011Co-Authors: Johan MazelAbstract:Anomaly detection has become a vital component of any network in today's Internet. Ranging from non-malicious unexpected events such as flash-crowds and failures, to network attacks such as denials-of-service and network scans, network traffic anomalies can have serious detrimental effects on the performance and integrity of the network. The continuous arising of new anomalies and attacks create a continuous challenge to cope with events that put the network integrity at risk. Moreover, the inner polymorphic nature of traffic caused, among other things, by a highly changing protocol landscape, complicates Anomaly detection system's task. In fact, most network Anomaly detection systems proposed so far employ knowledge-dependent techniques, using either misuse detection signature-based detection methods or Anomaly detection relying on supervisedlearning techniques. However, both approaches present major limitations: the former fails to detect and characterize unknown anomalies (letting the network unprotected for long periods) and the latter requires training over labeled normal traffic, which is a difficult and expensive stage that need to be updated on a regular basis to follow network traffic evolution. Such limitations impose a serious bottleneck to the previously presented problem. We introduce an unsupervised approach to detect and characterize network anomalies, without relying on signatures, statistical training, or labeled traffic, which represents a significant step towards the autonomy of networks. Unsupervised detection is accomplished by means of robust data-clustering techniques, combining Sub-Space clustering with Evidence Accumulation or Inter-Clustering Results Association, to blindly identify anomalies in traffic flows. Correlating the results of several unsupervised detections is also performed to improve detection robustness. The correlation results are further used along other Anomaly characteristics to build an Anomaly hierarchy in terms of dange rousness. Characterization is then achieved by building efficient filtering rules to describe a Detected Anomaly. The detection and characterization performances and sensitivities to parameters are evaluated over a substantial subset of the MAWI repository which contains real network traffic traces. Our work shows that unsupervised learning techniques allow Anomaly detection systems to isolate anomalous traffic without any previous knowledge. We think that this contribution constitutes a great step towards autonomous network Anomaly detection. This PhD thesis has been funded through the ECODE project by the European Commission under the Framework Programme 7. The goal of this project is to develop, implement, and validate experimentally a cognitive routing system that meet the challenges experienced by the Internet in terms of manageability and security, availability and accountability, as well as routing system scalability and quality. The concerned use case inside the ECODE project is network Anomaly detection.
-
sub space clustering inter clustering results association Anomaly correlation for unsupervised network Anomaly detection
Conference on Network and Service Management, 2011Co-Authors: Johan Mazel, Yann Labit, Pedro Casas, Philippe OwezarskiAbstract:Network Anomaly detection is a critical aspect of network management for instance for QoS, security, etc. The continuous arising of new anomalies and attacks create a continuous challenge to cope with events that put the network integrity at risk. Most network Anomaly detection systems proposed so far employ a supervised strategy to accomplish the task, using either signature-based detection methods or supervised-learning techniques. However, both approaches present major limitations: the former fails to detect and characterize unknown anomalies (letting the network unprotected for long periods), the latter requires training and labelled traffic, which is difficult and expensive to produce. Such limitations impose a serious bottleneck to the previously presented problem. We introduce an unsupervised approach to detect and characterize network anomalies, without relying on signatures, statistical training, or labelled traffic, which represents a significant step towards the autonomy of networks. Unsupervised detection is accomplished by means of robust data-clustering techniques, combining Sub-Space clustering with Evidence Accumulation or Inter-Clustering Results Association, to blindly identify anomalies in traffic flows. Correlating the results of the unsupervised detection is also performed for improving the detection robustness. Characterization is achieved by building efficient filtering rules to describe a Detected Anomaly. The detection and characterization performances of the unsupervised approach are evaluated on real network traffic.
-
sub space clustering and evidence accumulation for unsupervised network Anomaly detection
Traffic Monitoring and Analysis, 2011Co-Authors: Johan Mazel, Pedro Casas, Philippe OwezarskiAbstract:Network Anomaly detection has been a hot research topic for many years. Most detection systems proposed so far employ a supervised strategy to accomplish the task, using either signature-based detection methods or supervised-learning techniques. However, both approaches present major limitations: the former fails to detect unknown anomalies, the latter requires training and labeled traffic, which is difficult and expensive to produce. Such limitations impose a serious bottleneck to the development of novel and applicable methods in the near future network scenario, characterized by emerging applications and new variants of network attacks. This work introduces and evaluates an unsupervised approach to detect and characterize network anomalies, without relying on signatures, statistical training, or labeled traffic. Unsupervised detection is accomplished by means of robust data-clustering techniques, combining Sub-Space Clustering and multiple Evidence Accumulation algorithms to blindly identify anomalous traffic flows. Unsupervised characterization is achieved by exploring inter-flows structure from multiple outlooks, building filtering rules to describe a Detected Anomaly. Detection and characterization performance of the unsupervised approach is extensively evaluated with real traffic from two different data-sets: the public MAWI traffic repository, and the METROSEC project data-set. Obtained results show the viability of unsupervised network Anomaly detection and characterization, an ambitious goal so far unmet.
Philippe Owezarski - One of the best experts on this subject based on the ideXlab platform.
-
Sub-Space Clustering and Evidence Accumulation for Unsupervised Anomaly Detection in IP Networks
2016Co-Authors: Pedro Casas, Johan Mazel, Philippe Owezarski, Yann LabitAbstract:Network traffic Anomaly detection and analysis has been a hot research topic for many years. Current detection systems employ two different approaches to tackle the problem, even using signature-based detection methods or supervised machine-learning techniques. However, both approaches present serious ground limitations. The former fails to detect new unknown anomalies, the latter highly relies on labeled data for training, which is difficult and expensive to produce. These limitations become highly restrictive in current Internet traffic scenario, characterized by emerging network applications and new variants of network attacks. In this paper, we introduce a novel approach to detect network traffic attacks in a completely unsupervised fashion. The proposed method does not assume any Anomaly signature or particular model for Anomaly-free traffic, which allows for detection of previously unseen attacks. By combining the multiple evidence of traffic structure provided by sub-space clustering techniques, we show that our method can efficiently isolate and extract unknown anomalies buried inside large amounts of traffic. Apart from discovering new anomalies, the method automatically generates a new and easy-to-interpret signature for the novel Detected Anomaly, easing network administrator tasks. This new unsupervised Anomaly detection method is a powerful means to detect zero-day attacks in a changing environment, where signature-based or supervised learning may fail. We evaluate the ability of our promising proposal to discover a distributed attack in real traffic from the public MAWI traffic repository, discussing future directions and ongoing work.
-
sub space clustering inter clustering results association Anomaly correlation for unsupervised network Anomaly detection
Conference on Network and Service Management, 2011Co-Authors: Johan Mazel, Yann Labit, Pedro Casas, Philippe OwezarskiAbstract:Network Anomaly detection is a critical aspect of network management for instance for QoS, security, etc. The continuous arising of new anomalies and attacks create a continuous challenge to cope with events that put the network integrity at risk. Most network Anomaly detection systems proposed so far employ a supervised strategy to accomplish the task, using either signature-based detection methods or supervised-learning techniques. However, both approaches present major limitations: the former fails to detect and characterize unknown anomalies (letting the network unprotected for long periods), the latter requires training and labelled traffic, which is difficult and expensive to produce. Such limitations impose a serious bottleneck to the previously presented problem. We introduce an unsupervised approach to detect and characterize network anomalies, without relying on signatures, statistical training, or labelled traffic, which represents a significant step towards the autonomy of networks. Unsupervised detection is accomplished by means of robust data-clustering techniques, combining Sub-Space clustering with Evidence Accumulation or Inter-Clustering Results Association, to blindly identify anomalies in traffic flows. Correlating the results of the unsupervised detection is also performed for improving the detection robustness. Characterization is achieved by building efficient filtering rules to describe a Detected Anomaly. The detection and characterization performances of the unsupervised approach are evaluated on real network traffic.
-
sub space clustering and evidence accumulation for unsupervised network Anomaly detection
Traffic Monitoring and Analysis, 2011Co-Authors: Johan Mazel, Pedro Casas, Philippe OwezarskiAbstract:Network Anomaly detection has been a hot research topic for many years. Most detection systems proposed so far employ a supervised strategy to accomplish the task, using either signature-based detection methods or supervised-learning techniques. However, both approaches present major limitations: the former fails to detect unknown anomalies, the latter requires training and labeled traffic, which is difficult and expensive to produce. Such limitations impose a serious bottleneck to the development of novel and applicable methods in the near future network scenario, characterized by emerging applications and new variants of network attacks. This work introduces and evaluates an unsupervised approach to detect and characterize network anomalies, without relying on signatures, statistical training, or labeled traffic. Unsupervised detection is accomplished by means of robust data-clustering techniques, combining Sub-Space Clustering and multiple Evidence Accumulation algorithms to blindly identify anomalous traffic flows. Unsupervised characterization is achieved by exploring inter-flows structure from multiple outlooks, building filtering rules to describe a Detected Anomaly. Detection and characterization performance of the unsupervised approach is extensively evaluated with real traffic from two different data-sets: the public MAWI traffic repository, and the METROSEC project data-set. Obtained results show the viability of unsupervised network Anomaly detection and characterization, an ambitious goal so far unmet.
Pedro Casas - One of the best experts on this subject based on the ideXlab platform.
-
Sub-Space Clustering and Evidence Accumulation for Unsupervised Anomaly Detection in IP Networks
2016Co-Authors: Pedro Casas, Johan Mazel, Philippe Owezarski, Yann LabitAbstract:Network traffic Anomaly detection and analysis has been a hot research topic for many years. Current detection systems employ two different approaches to tackle the problem, even using signature-based detection methods or supervised machine-learning techniques. However, both approaches present serious ground limitations. The former fails to detect new unknown anomalies, the latter highly relies on labeled data for training, which is difficult and expensive to produce. These limitations become highly restrictive in current Internet traffic scenario, characterized by emerging network applications and new variants of network attacks. In this paper, we introduce a novel approach to detect network traffic attacks in a completely unsupervised fashion. The proposed method does not assume any Anomaly signature or particular model for Anomaly-free traffic, which allows for detection of previously unseen attacks. By combining the multiple evidence of traffic structure provided by sub-space clustering techniques, we show that our method can efficiently isolate and extract unknown anomalies buried inside large amounts of traffic. Apart from discovering new anomalies, the method automatically generates a new and easy-to-interpret signature for the novel Detected Anomaly, easing network administrator tasks. This new unsupervised Anomaly detection method is a powerful means to detect zero-day attacks in a changing environment, where signature-based or supervised learning may fail. We evaluate the ability of our promising proposal to discover a distributed attack in real traffic from the public MAWI traffic repository, discussing future directions and ongoing work.
-
sub space clustering inter clustering results association Anomaly correlation for unsupervised network Anomaly detection
Conference on Network and Service Management, 2011Co-Authors: Johan Mazel, Yann Labit, Pedro Casas, Philippe OwezarskiAbstract:Network Anomaly detection is a critical aspect of network management for instance for QoS, security, etc. The continuous arising of new anomalies and attacks create a continuous challenge to cope with events that put the network integrity at risk. Most network Anomaly detection systems proposed so far employ a supervised strategy to accomplish the task, using either signature-based detection methods or supervised-learning techniques. However, both approaches present major limitations: the former fails to detect and characterize unknown anomalies (letting the network unprotected for long periods), the latter requires training and labelled traffic, which is difficult and expensive to produce. Such limitations impose a serious bottleneck to the previously presented problem. We introduce an unsupervised approach to detect and characterize network anomalies, without relying on signatures, statistical training, or labelled traffic, which represents a significant step towards the autonomy of networks. Unsupervised detection is accomplished by means of robust data-clustering techniques, combining Sub-Space clustering with Evidence Accumulation or Inter-Clustering Results Association, to blindly identify anomalies in traffic flows. Correlating the results of the unsupervised detection is also performed for improving the detection robustness. Characterization is achieved by building efficient filtering rules to describe a Detected Anomaly. The detection and characterization performances of the unsupervised approach are evaluated on real network traffic.
-
sub space clustering and evidence accumulation for unsupervised network Anomaly detection
Traffic Monitoring and Analysis, 2011Co-Authors: Johan Mazel, Pedro Casas, Philippe OwezarskiAbstract:Network Anomaly detection has been a hot research topic for many years. Most detection systems proposed so far employ a supervised strategy to accomplish the task, using either signature-based detection methods or supervised-learning techniques. However, both approaches present major limitations: the former fails to detect unknown anomalies, the latter requires training and labeled traffic, which is difficult and expensive to produce. Such limitations impose a serious bottleneck to the development of novel and applicable methods in the near future network scenario, characterized by emerging applications and new variants of network attacks. This work introduces and evaluates an unsupervised approach to detect and characterize network anomalies, without relying on signatures, statistical training, or labeled traffic. Unsupervised detection is accomplished by means of robust data-clustering techniques, combining Sub-Space Clustering and multiple Evidence Accumulation algorithms to blindly identify anomalous traffic flows. Unsupervised characterization is achieved by exploring inter-flows structure from multiple outlooks, building filtering rules to describe a Detected Anomaly. Detection and characterization performance of the unsupervised approach is extensively evaluated with real traffic from two different data-sets: the public MAWI traffic repository, and the METROSEC project data-set. Obtained results show the viability of unsupervised network Anomaly detection and characterization, an ambitious goal so far unmet.
Vivek Srikumar - One of the best experts on this subject based on the ideXlab platform.
-
ACM Conference on Computer and Communications Security - DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning
Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security - CCS '17, 2017Co-Authors: Min Du, Guineng Zheng, Feifei Li, Vivek SrikumarAbstract:Anomaly detection is a critical step towards building a secure and trustworthy system. The primary purpose of a system log is to record system states and significant events at various critical points to help debug system failures and perform root cause analysis. Such log data is universally available in nearly all computer systems. Log data is an important and valuable resource for understanding system status and performance issues; therefore, the various system logs are naturally excellent source of information for online monitoring and Anomaly detection. We propose DeepLog, a deep neural network model utilizing Long Short-Term Memory (LSTM), to model a system log as a natural language sequence. This allows DeepLog to automatically learn log patterns from normal execution, and detect anomalies when log patterns deviate from the model trained from log data under normal execution. In addition, we demonstrate how to incrementally update the DeepLog model in an online fashion so that it can adapt to new log patterns over time. Furthermore, DeepLog constructs workflows from the underlying system log so that once an Anomaly is Detected, users can diagnose the Detected Anomaly and perform root cause analysis effectively. Extensive experimental evaluations over large log data have shown that DeepLog has outperformed other existing log-based Anomaly detection methods based on traditional data mining methodologies.
-
deeplog Anomaly detection and diagnosis from system logs through deep learning
Computer and Communications Security, 2017Co-Authors: Min Du, Guineng Zheng, Feifei Li, Vivek SrikumarAbstract:Anomaly detection is a critical step towards building a secure and trustworthy system. The primary purpose of a system log is to record system states and significant events at various critical points to help debug system failures and perform root cause analysis. Such log data is universally available in nearly all computer systems. Log data is an important and valuable resource for understanding system status and performance issues; therefore, the various system logs are naturally excellent source of information for online monitoring and Anomaly detection. We propose DeepLog, a deep neural network model utilizing Long Short-Term Memory (LSTM), to model a system log as a natural language sequence. This allows DeepLog to automatically learn log patterns from normal execution, and detect anomalies when log patterns deviate from the model trained from log data under normal execution. In addition, we demonstrate how to incrementally update the DeepLog model in an online fashion so that it can adapt to new log patterns over time. Furthermore, DeepLog constructs workflows from the underlying system log so that once an Anomaly is Detected, users can diagnose the Detected Anomaly and perform root cause analysis effectively. Extensive experimental evaluations over large log data have shown that DeepLog has outperformed other existing log-based Anomaly detection methods based on traditional data mining methodologies.
Pascal Poncelet - One of the best experts on this subject based on the ideXlab platform.
-
Anomaly Detection in Monitoring Sensor Data for Preventive Maintenance
Expert Systems with Applications, 2011Co-Authors: Julien Rabatel, Sandra Bringay, Pascal PonceletAbstract:Today, many industrial companies must face problems raised by maintenance. In particular, the Anomaly detection problem is probably one of the most challenging. In this paper we focus on the railway maintenance task and propose to automatically detect anomalies in order to predict in advance potential failures. We first address the problem of characterizing normal behavior. In order to extract interesting patterns, we have developed a method to take into account the contextual criteria associated to railway data (itinerary, weather conditions, etc.). We then measure the compliance of new data, according to extracted knowledge, and provide information about the seriousness and the exact localization of a Detected Anomaly.
-
SO_MAD: SensOr Mining for Anomaly Detection in Railway Data
2009Co-Authors: Julien Rabatel, Sandra Bringay, Pascal PonceletAbstract:Today, many industrial companies must face problems raised by maintenance. In particular, the Anomaly detection problem is probably one of the most challenging. In this paper we focus on the railway maintenance task and propose to automatically detect anomalies in order to predict in advance potential failures. We first address the problem of characterizing normal behavior. In order to extract interesting patterns, we have developed a method to take into account the contextual criteria associated to railway data (itinerary, weather conditions, etc.). We then measure the compliance of new data, according to extracted knowledge, and provide information about the seriousness and possible causes of a Detected Anomaly.