The Experts below are selected from a list of 2226 Experts worldwide ranked by ideXlab platform
Marco Vieira - One of the best experts on this subject based on the ideXlab platform.
-
On the Metrics for Benchmarking Vulnerability Detection Tools
2015 45th Annual IEEE IFIP International Conference on Dependable Systems and Networks, 2015Co-Authors: Nuno Antunes, Marco VieiraAbstract:Research and practice show that the effectiveness of Vulnerability Detection tools depends on the concrete use scenario. Benchmarking can be used for selecting the most appropriate tool, helping assessing and comparing alternative solutions, but its effectiveness largely depends on the adequacy of the metrics. This paper studies the problem of selecting the metrics to be used in a benchmark for software Vulnerability Detection tools. First, a large set of metrics is gathered and analyzed according to the characteristics of a good metric for the Vulnerability Detection domain. Afterwards, the metrics are analyzed in the context of specific Vulnerability Detection scenarios to understand their effectiveness and to select the most adequate one for each scenario. Finally, an MCDA algorithm together with experts' judgment is applied to validate the conclusions. Results show that although some of the metrics traditionally used like precision and recall are adequate in some scenarios, others require alternative metrics that are seldom used in the benchmarking area.
-
DSN - On the Metrics for Benchmarking Vulnerability Detection Tools
2015 45th Annual IEEE IFIP International Conference on Dependable Systems and Networks, 2015Co-Authors: Nuno Antunes, Marco VieiraAbstract:Research and practice show that the effectiveness of Vulnerability Detection tools depends on the concrete use scenario. Benchmarking can be used for selecting the most appropriate tool, helping assessing and comparing alternative solutions, but its effectiveness largely depends on the adequacy of the metrics. This paper studies the problem of selecting the metrics to be used in a benchmark for software Vulnerability Detection tools. First, a large set of metrics is gathered and analyzed according to the characteristics of a good metric for the Vulnerability Detection domain. Afterwards, the metrics are analyzed in the context of specific Vulnerability Detection scenarios to understand their effectiveness and to select the most adequate one for each scenario. Finally, an MCDA algorithm together with experts' judgment is applied to validate the conclusions. Results show that although some of the metrics traditionally used like precision and recall are adequate in some scenarios, others require alternative metrics that are seldom used in the benchmarking area.
-
Assessing and Comparing Vulnerability Detection Tools for Web Services: Benchmarking Approach and Examples
IEEE Transactions on Services Computing, 2015Co-Authors: Nuno Antunes, Marco VieiraAbstract:Selecting a Vulnerability Detection tool is a key problem that is frequently faced by developers of security-critical web services. Research and practice shows that state-of-the-art tools present low effectiveness both in terms of Vulnerability coverage and false positive rates. The main problem is that such tools are typically limited in the Detection approaches implemented, and are designed for being applied in very concrete scenarios. Thus, using the wrong tool may lead to the deployment of services with undetected vulnerabilities. This paper proposes a benchmarking approach to assess and compare the effectiveness of Vulnerability Detection tools in web services environments. This approach was used to define two concrete benchmarks for SQL Injection Vulnerability Detection tools. The first is based on a predefined set of web services, and the second allows the benchmark user to specify the workload that best portrays the specific characteristics of his environment. The two benchmarks are used to assess and compare several widely used tools, including four penetration testers, three static code analyzers, and one anomaly detector. Results show that the benchmarks accurately portray the effectiveness of Vulnerability Detection tools (in a relative manner) and suggest that the proposed benchmarking approach can be applied in the field.
-
Benchmarking Vulnerability Detection Tools for Web Services
2010 IEEE International Conference on Web Services, 2010Co-Authors: Nuno Antunes, Marco VieiraAbstract:Vulnerability Detection tools are frequently considered the silver-bullet for detecting vulnerabilities in web services. However, research shows that the effectiveness of most of those tools is very low and that using the wrong tool may lead to the deployment of services with undetected vulnerabilities. In this paper we propose a benchmarking approach to assess and compare the effectiveness of Vulnerability Detection tools in web services environments. This approach was used to define a concrete benchmark for SQL Injection Vulnerability Detection tools. This benchmark is demonstrated by a real example of benchmarking several widely used tools, including four penetration-testers, three static code analyzers, and one anomaly detector. Results show that the benchmark accurately portrays the effectiveness of Vulnerability Detection tools and suggest that the proposed approach can be applied in the field.
-
ICWS - Benchmarking Vulnerability Detection Tools for Web Services
2010 IEEE International Conference on Web Services, 2010Co-Authors: Nuno Antunes, Marco VieiraAbstract:Vulnerability Detection tools are frequently considered the silver-bullet for detecting vulnerabilities in web services. However, research shows that the effectiveness of most of those tools is very low and that using the wrong tool may lead to the deployment of services with undetected vulnerabilities. In this paper we propose a benchmarking approach to assess and compare the effectiveness of Vulnerability Detection tools in web services environments. This approach was used to define a concrete benchmark for SQL Injection Vulnerability Detection tools. This benchmark is demonstrated by a real example of benchmarking several widely used tools, including four penetration-testers, three static code analyzers, and one anomaly detector. Results show that the benchmark accurately portrays the effectiveness of Vulnerability Detection tools and suggest that the proposed approach can be applied in the field.
Nuno Antunes - One of the best experts on this subject based on the ideXlab platform.
-
On the Metrics for Benchmarking Vulnerability Detection Tools
2015 45th Annual IEEE IFIP International Conference on Dependable Systems and Networks, 2015Co-Authors: Nuno Antunes, Marco VieiraAbstract:Research and practice show that the effectiveness of Vulnerability Detection tools depends on the concrete use scenario. Benchmarking can be used for selecting the most appropriate tool, helping assessing and comparing alternative solutions, but its effectiveness largely depends on the adequacy of the metrics. This paper studies the problem of selecting the metrics to be used in a benchmark for software Vulnerability Detection tools. First, a large set of metrics is gathered and analyzed according to the characteristics of a good metric for the Vulnerability Detection domain. Afterwards, the metrics are analyzed in the context of specific Vulnerability Detection scenarios to understand their effectiveness and to select the most adequate one for each scenario. Finally, an MCDA algorithm together with experts' judgment is applied to validate the conclusions. Results show that although some of the metrics traditionally used like precision and recall are adequate in some scenarios, others require alternative metrics that are seldom used in the benchmarking area.
-
DSN - On the Metrics for Benchmarking Vulnerability Detection Tools
2015 45th Annual IEEE IFIP International Conference on Dependable Systems and Networks, 2015Co-Authors: Nuno Antunes, Marco VieiraAbstract:Research and practice show that the effectiveness of Vulnerability Detection tools depends on the concrete use scenario. Benchmarking can be used for selecting the most appropriate tool, helping assessing and comparing alternative solutions, but its effectiveness largely depends on the adequacy of the metrics. This paper studies the problem of selecting the metrics to be used in a benchmark for software Vulnerability Detection tools. First, a large set of metrics is gathered and analyzed according to the characteristics of a good metric for the Vulnerability Detection domain. Afterwards, the metrics are analyzed in the context of specific Vulnerability Detection scenarios to understand their effectiveness and to select the most adequate one for each scenario. Finally, an MCDA algorithm together with experts' judgment is applied to validate the conclusions. Results show that although some of the metrics traditionally used like precision and recall are adequate in some scenarios, others require alternative metrics that are seldom used in the benchmarking area.
-
Assessing and Comparing Vulnerability Detection Tools for Web Services: Benchmarking Approach and Examples
IEEE Transactions on Services Computing, 2015Co-Authors: Nuno Antunes, Marco VieiraAbstract:Selecting a Vulnerability Detection tool is a key problem that is frequently faced by developers of security-critical web services. Research and practice shows that state-of-the-art tools present low effectiveness both in terms of Vulnerability coverage and false positive rates. The main problem is that such tools are typically limited in the Detection approaches implemented, and are designed for being applied in very concrete scenarios. Thus, using the wrong tool may lead to the deployment of services with undetected vulnerabilities. This paper proposes a benchmarking approach to assess and compare the effectiveness of Vulnerability Detection tools in web services environments. This approach was used to define two concrete benchmarks for SQL Injection Vulnerability Detection tools. The first is based on a predefined set of web services, and the second allows the benchmark user to specify the workload that best portrays the specific characteristics of his environment. The two benchmarks are used to assess and compare several widely used tools, including four penetration testers, three static code analyzers, and one anomaly detector. Results show that the benchmarks accurately portray the effectiveness of Vulnerability Detection tools (in a relative manner) and suggest that the proposed benchmarking approach can be applied in the field.
-
Benchmarking Vulnerability Detection Tools for Web Services
2010 IEEE International Conference on Web Services, 2010Co-Authors: Nuno Antunes, Marco VieiraAbstract:Vulnerability Detection tools are frequently considered the silver-bullet for detecting vulnerabilities in web services. However, research shows that the effectiveness of most of those tools is very low and that using the wrong tool may lead to the deployment of services with undetected vulnerabilities. In this paper we propose a benchmarking approach to assess and compare the effectiveness of Vulnerability Detection tools in web services environments. This approach was used to define a concrete benchmark for SQL Injection Vulnerability Detection tools. This benchmark is demonstrated by a real example of benchmarking several widely used tools, including four penetration-testers, three static code analyzers, and one anomaly detector. Results show that the benchmark accurately portrays the effectiveness of Vulnerability Detection tools and suggest that the proposed approach can be applied in the field.
-
ICWS - Benchmarking Vulnerability Detection Tools for Web Services
2010 IEEE International Conference on Web Services, 2010Co-Authors: Nuno Antunes, Marco VieiraAbstract:Vulnerability Detection tools are frequently considered the silver-bullet for detecting vulnerabilities in web services. However, research shows that the effectiveness of most of those tools is very low and that using the wrong tool may lead to the deployment of services with undetected vulnerabilities. In this paper we propose a benchmarking approach to assess and compare the effectiveness of Vulnerability Detection tools in web services environments. This approach was used to define a concrete benchmark for SQL Injection Vulnerability Detection tools. This benchmark is demonstrated by a real example of benchmarking several widely used tools, including four penetration-testers, three static code analyzers, and one anomaly detector. Results show that the benchmark accurately portrays the effectiveness of Vulnerability Detection tools and suggest that the proposed approach can be applied in the field.
Zhen Li - One of the best experts on this subject based on the ideXlab platform.
-
A Comparative Study of Deep Learning-Based Vulnerability Detection System
IEEE Access, 2019Co-Authors: Zhen Li, Jing Tang, Zhihao ZhangAbstract:Source code static analysis has been widely used to detect vulnerabilities in the development of software products. The Vulnerability patterns purely based on human experts are laborious and error prone, which has motivated the use of machine learning for Vulnerability Detection. In order to relieve human experts of defining Vulnerability rules or features, a recent study shows the feasibility of leveraging deep learning to detect vulnerabilities automatically. However, the impact of different factors on the effectiveness of Vulnerability Detection is unknown. In this paper, we collect two datasets from the programs involving 126 types of vulnerabilities, on which we conduct the first comparative study to quantitatively evaluate the impact of different factors on the effectiveness of Vulnerability Detection. The experimental results show that accommodating control dependency can increase the overall effectiveness of Vulnerability Detection F1-measure by 20.3%; the imbalanced data processing methods are not effective for the dataset we create; bidirectional recurrent neural networks (RNNs) are more effective than unidirectional RNNs and convolutional neural network, which in turn are more effective than multi-layer perception; using the last output corresponding to the time step for the bidirectional long short-term memory (BLSTM) can reduce the false negative rate by 2.0% at the price of increasing the false positive rate by 0.5%.
-
vuldeepecker a deep learning based system for Vulnerability Detection
Network and Distributed System Security Symposium, 2018Co-Authors: Zhen Li, Shouhuai Xu, Xinyu Ou, Sujuan Wang, Zhijun Deng, Yuyi ZhongAbstract:The automatic Detection of software vulnerabilities is an important research problem. However, existing solutions to this problem rely on human experts to define features and often miss many vulnerabilities (i.e., incurring high false negative rate). In this paper, we initiate the study of using deep learning-based Vulnerability Detection to relieve human experts from the tedious and subjective task of manually defining features. Since deep learning is motivated to deal with problems that are very different from the problem of Vulnerability Detection, we need some guiding principles for applying deep learning to Vulnerability Detection. In particular, we need to find representations of software programs that are suitable for deep learning. For this purpose, we propose using code gadgets to represent programs and then transform them into vectors, where a code gadget is a number of (not necessarily consecutive) lines of code that are semantically related to each other. This leads to the design and implementation of a deep learning-based Vulnerability Detection system, called Vulnerability Deep Pecker (VulDeePecker). In order to evaluate VulDeePecker, we present the first Vulnerability dataset for deep learning approaches. Experimental results show that VulDeePecker can achieve much fewer false negatives (with reasonable false positives) than other approaches. We further apply VulDeePecker to 3 software products (namely Xen, Seamonkey, and Libav) and detect 4 vulnerabilities, which are not reported in the National Vulnerability Database but were "silently" patched by the vendors when releasing later versions of these products; in contrast, these vulnerabilities are almost entirely missed by the other Vulnerability Detection systems we experimented with.
-
NDSS - VulDeePecker: A Deep Learning-Based System for Vulnerability Detection.
Proceedings 2018 Network and Distributed System Security Symposium, 2018Co-Authors: Zhen Li, Shouhuai Xu, Xinyu Ou, Sujuan Wang, Zhijun Deng, Yuyi ZhongAbstract:The automatic Detection of software vulnerabilities is an important research problem. However, existing solutions to this problem rely on human experts to define features and often miss many vulnerabilities (i.e., incurring high false negative rate). In this paper, we initiate the study of using deep learning-based Vulnerability Detection to relieve human experts from the tedious and subjective task of manually defining features. Since deep learning is motivated to deal with problems that are very different from the problem of Vulnerability Detection, we need some guiding principles for applying deep learning to Vulnerability Detection. In particular, we need to find representations of software programs that are suitable for deep learning. For this purpose, we propose using code gadgets to represent programs and then transform them into vectors, where a code gadget is a number of (not necessarily consecutive) lines of code that are semantically related to each other. This leads to the design and implementation of a deep learning-based Vulnerability Detection system, called Vulnerability Deep Pecker (VulDeePecker). In order to evaluate VulDeePecker, we present the first Vulnerability dataset for deep learning approaches. Experimental results show that VulDeePecker can achieve much fewer false negatives (with reasonable false positives) than other approaches. We further apply VulDeePecker to 3 software products (namely Xen, Seamonkey, and Libav) and detect 4 vulnerabilities, which are not reported in the National Vulnerability Database but were "silently" patched by the vendors when releasing later versions of these products; in contrast, these vulnerabilities are almost entirely missed by the other Vulnerability Detection systems we experimented with.
-
μVulDeePecker: A Deep Learning-Based System for Multiclass Vulnerability Detection
IEEE Transactions on Dependable and Secure Computing, 1Co-Authors: Sujuan Wang, Shouhuai Xu, Zhen LiAbstract:Fine-grained software Vulnerability Detection is an important and challenging problem. Ideally, a Detection system (or detector) not only should be able to detect whether or not a program contains vulnerabilities, but also should be able to pinpoint the type of a Vulnerability in question. Existing Vulnerability Detection methods based on deep learning can detect the presence of vulnerabilities (i.e., addressing the binary classification or Detection problem), but cannot pinpoint types of vulnerabilities (i.e., incapable of addressing multiclass classification). In this paper, we propose the first deep learning-based system for multiclass Vulnerability Detection, dubbed μVulDeePecker. The key insight underlying μVulDeePecker is the concept of code attention, which can capture information that can help pinpoint types of vulnerabilities, even when the samples are small. For this purpose, we create a dataset from scratch and use it to evaluate the effectiveness of μVulDeePecker. Experimental results show that μVulDeePecker is effective for multiclass Vulnerability Detection and that accommodating control-dependence (other than data-dependence) can lead to higher Detection capabilities.
Chaojing Tang - One of the best experts on this subject based on the ideXlab platform.
-
Test Case Generation for Vulnerability Detection Using Genetic Algorithm
Proceedings of the 2015 4th National Conference on Electrical Electronics and Computer Engineering, 2020Co-Authors: Bo Shuai, Haifeng Li, Quan Zhang, Jian Wang, Chaojing TangAbstract:In order to elevate efficiency of traditional Fuzzing technique, a novel method using genetic algorithm is proposed based on path coverage and test cost. There are evidences that GA has been already successful in generating test cases. Considering path coverage as the test adequacy criterion, we have designed a GA-based test data generator that is able to synthesize multiple test data to cover multiple target paths. Meanwhile, in order to reduce the test cost in Fuzzing process, test cost is analyzed respectively from running time and loop structure in the method. Experimental results show that proposed approach could obtain higher Vulnerability Detection accuracy and efficiency.
-
CIS - Software Vulnerability Detection Based on Code Coverage and Test Cost
2015 11th International Conference on Computational Intelligence and Security (CIS), 2015Co-Authors: Bo Shuai, Haifeng Li, Quan Zhang, Lei Zhang, Chaojing TangAbstract:In order to solve the problems of traditional Fuzzing technique for software Vulnerability Detection, a novel method based on code coverage and test cost is proposed. Firstly, static analysis is applied to calculate the code coverage information, including basic block coverage and new block coverage. In addition, test path diversity information is introduced to elevate path coverage, which is achieved based on the sequence alignment algorithm. Secondly, test cost is analyzed respectively from running time and loop structure. The loop structure is simplified using finite expansion manner. Thirdly, the genetic algorithm fitness function is constructed based on the code coverage and test cost to guide the test case generation. Experiments on realistic binary software show that the method could obtain higher Vulnerability Detection accuracy and efficiency than the traditional Fuzzing technique.
-
Software Vulnerability Detection Based on Code Coverage and Test Cost
2015 11th International Conference on Computational Intelligence and Security (CIS), 2015Co-Authors: Bo Shuai, Haifeng Li, Quan Zhang, Lei Zhang, Chaojing TangAbstract:In order to solve the problems of traditional Fuzzing technique for software Vulnerability Detection, a novel method based on code coverage and test cost is proposed. Firstly, static analysis is applied to calculate the code coverage information, including basic block coverage and new block coverage. In addition, test path diversity information is introduced to elevate path coverage, which is achieved based on the sequence alignment algorithm. Secondly, test cost is analyzed respectively from running time and loop structure. The loop structure is simplified using finite expansion manner. Thirdly, the genetic algorithm fitness function is constructed based on the code coverage and test cost to guide the test case generation. Experiments on realistic binary software show that the method could obtain higher Vulnerability Detection accuracy and efficiency than the traditional Fuzzing technique.
-
Software Vulnerability Detection using genetic algorithm and dynamic taint analysis
2013 3rd International Conference on Consumer Electronics Communications and Networks, 2013Co-Authors: Bo Shuai, Mengjun Li, Haifeng Li, Quan Zhang, Chaojing TangAbstract:In order to solve the problems of traditional Fuzzing technique for software Vulnerability Detection, this paper proposes a novel method based on genetic algorithm and dynamic taint analysis. First, static analysis is applied to calculate the critical path information, including danger functions, high cyclomatic number functions and loop structures. Second, dynamic taint analysis is introduced to identify the key bytes to reduce the input space. Third, the genetic algorithm fitness function is constructed based on the critical path information to guide the test case generation and the genetic operators are executed on the reduced input space. Experiments show that the method could obtain higher Vulnerability Detection accuracy and efficiency.
Lionel Briand - One of the best experts on this subject based on the ideXlab platform.
-
Search-Driven String Constraint Solving for Vulnerability Detection
2017 IEEE ACM 39th International Conference on Software Engineering (ICSE), 2017Co-Authors: Julian Thomé, Lwin Khin Shar, Domenico Bianculli, Lionel BriandAbstract:Constraint solving is an essential technique for detecting vulnerabilities in programs, since it can reason about input sanitization and validation operations performed on user inputs. However, real-world programs typically contain complex string operations that challenge Vulnerability Detection. State-of-the-art string constraint solvers support only a limited set of string operations and fail when they encounter an unsupported one, this leads to limited effectiveness in finding vulnerabilities. In this paper we propose a search-driven constraint solving technique that complements the support for complex string operations provided by any existing string constraint solver. Our technique uses a hybrid constraint solving procedure based on the Ant Colony Optimization meta-heuristic. The idea is to execute it as a fallback mechanism, only when a solver encounters a constraint containing an operation that it does not support. We have implemented the proposed search-driven constraint solving technique in the ACO-Solver tool, which we have evaluated in the context of injection and XSS Vulnerability Detection for Java Web applications. We have assessed the benefits and costs of combining the proposed technique with two state-of-the-art constraint solvers (Z3-str2 and CVC4). The experimental results, based on a benchmark with 104 constraints derived from nine realistic Web applications, show that our approach, when combined in a state-of-the-art solver, significantly improves the number of detected vulnerabilities (from 4.7% to 71.9% for Z3-str2, from 85.9% to 100.0% for CVC4), and solves several cases on which the solver fails when used stand-alone (46 more solved cases for Z3-str2, and 11 more for CVC4), while still keeping the execution time affordable in practice.
