Benchmarking Approach - Explore the Science & Experts | ideXlab

Scan Science and Technology

Contact Leading Edge Experts & Companies

Benchmarking Approach

The Experts below are selected from a list of 3546 Experts worldwide ranked by ideXlab platform

Marco Vieira – 1st expert on this subject based on the ideXlab platform

  • Assessing and Comparing Vulnerability Detection Tools for Web Services: Benchmarking Approach and Examples
    IEEE Transactions on Services Computing, 2015
    Co-Authors: Nuno Antunes, Marco Vieira

    Abstract:

    Selecting a vulnerability detection tool is a key problem that is frequently faced by developers of security-critical web services. Research and practice shows that state-of-the-art tools present low effectiveness both in terms of vulnerability coverage and false positive rates. The main problem is that such tools are typically limited in the detection Approaches implemented, and are designed for being applied in very concrete scenarios. Thus, using the wrong tool may lead to the deployment of services with undetected vulnerabilities. This paper proposes a Benchmarking Approach to assess and compare the effectiveness of vulnerability detection tools in web services environments. This Approach was used to define two concrete benchmarks for SQL Injection vulnerability detection tools. The first is based on a predefined set of web services, and the second allows the benchmark user to specify the workload that best portrays the specific characteristics of his environment. The two benchmarks are used to assess and compare several widely used tools, including four penetration testers, three static code analyzers, and one anomaly detector. Results show that the benchmarks accurately portray the effectiveness of vulnerability detection tools (in a relative manner) and suggest that the proposed Benchmarking Approach can be applied in the field.

  • Detecting Vulnerabilities in Service Oriented Architectures
    2012 IEEE 23rd International Symposium on Software Reliability Engineering Workshops, 2012
    Co-Authors: Nuno Antunes, Marco Vieira

    Abstract:

    The adoption of Service Oriented Architectures (SOAs) in a wide range of organizations, including business-critical systems, opens the door to new security challenges. Although the services used should be secure and reliable, they are often deployed with security bugs that can be maliciously exploited. The problem is that developers are frequently not specialized on security and the common time-to-market constraints limits an in depth test for vulnerabilities. Additionally, research and practice shows that the effectiveness of existing vulnerability detection tools is very poor. The goal of this work is to advance the state-of-the-art by investigating new techniques and tools to effectively detect vulnerabilities in SOAs in an automated manner. Instrumental in this work is to propose a Benchmarking Approach that allows assessing and comparing vulnerability detection tools, thus helping guiding tools development and improvement, and allowing users to select the most effective ones according to specific needs.

  • TO BEnchmark or NOT TO BEnchmark security: That is the question
    2011 IEEE IFIP 41st International Conference on Dependable Systems and Networks Workshops (DSN-W), 2011
    Co-Authors: Afonso Araújo Neto, Marco Vieira

    Abstract:

    The multiplicity of available software and component alternatives has boosted the interest in suitable benchmarks, able to assist in the selection of candidate solutions from the existing diversity, concerning several attributes. The huge success of performance and dependability Benchmarking, however, markedly contrasts with the small advances on security Benchmarking, which has only sparsely been studied in recent years. In this position paper we discuss the difficulties involved in applying the dependability Benchmarking Approach to the security context, and propose and discuss an appealing alternative: trustworthiness Benchmarking.

Nuno Antunes – 2nd expert on this subject based on the ideXlab platform

  • Assessing and Comparing Vulnerability Detection Tools for Web Services: Benchmarking Approach and Examples
    IEEE Transactions on Services Computing, 2015
    Co-Authors: Nuno Antunes, Marco Vieira

    Abstract:

    Selecting a vulnerability detection tool is a key problem that is frequently faced by developers of security-critical web services. Research and practice shows that state-of-the-art tools present low effectiveness both in terms of vulnerability coverage and false positive rates. The main problem is that such tools are typically limited in the detection Approaches implemented, and are designed for being applied in very concrete scenarios. Thus, using the wrong tool may lead to the deployment of services with undetected vulnerabilities. This paper proposes a Benchmarking Approach to assess and compare the effectiveness of vulnerability detection tools in web services environments. This Approach was used to define two concrete benchmarks for SQL Injection vulnerability detection tools. The first is based on a predefined set of web services, and the second allows the benchmark user to specify the workload that best portrays the specific characteristics of his environment. The two benchmarks are used to assess and compare several widely used tools, including four penetration testers, three static code analyzers, and one anomaly detector. Results show that the benchmarks accurately portray the effectiveness of vulnerability detection tools (in a relative manner) and suggest that the proposed Benchmarking Approach can be applied in the field.

  • Detecting Vulnerabilities in Service Oriented Architectures
    2012 IEEE 23rd International Symposium on Software Reliability Engineering Workshops, 2012
    Co-Authors: Nuno Antunes, Marco Vieira

    Abstract:

    The adoption of Service Oriented Architectures (SOAs) in a wide range of organizations, including business-critical systems, opens the door to new security challenges. Although the services used should be secure and reliable, they are often deployed with security bugs that can be maliciously exploited. The problem is that developers are frequently not specialized on security and the common time-to-market constraints limits an in depth test for vulnerabilities. Additionally, research and practice shows that the effectiveness of existing vulnerability detection tools is very poor. The goal of this work is to advance the state-of-the-art by investigating new techniques and tools to effectively detect vulnerabilities in SOAs in an automated manner. Instrumental in this work is to propose a Benchmarking Approach that allows assessing and comparing vulnerability detection tools, thus helping guiding tools development and improvement, and allowing users to select the most effective ones according to specific needs.

  • Benchmarking Vulnerability Detection Tools for Web Services
    2010 IEEE International Conference on Web Services, 2010
    Co-Authors: Nuno Antunes, Marco Vieira

    Abstract:

    Vulnerability detection tools are frequently considered the silver-bullet for detecting vulnerabilities in web services. However, research shows that the effectiveness of most of those tools is very low and that using the wrong tool may lead to the deployment of services with undetected vulnerabilities. In this paper we propose a Benchmarking Approach to assess and compare the effectiveness of vulnerability detection tools in web services environments. This Approach was used to define a concrete benchmark for SQL Injection vulnerability detection tools. This benchmark is demonstrated by a real example of Benchmarking several widely used tools, including four penetration-testers, three static code analyzers, and one anomaly detector. Results show that the benchmark accurately portrays the effectiveness of vulnerability detection tools and suggest that the proposed Approach can be applied in the field.

Matin Mohaghegh – 3rd expert on this subject based on the ideXlab platform

  • SOLI – Raw material flow optimization as a capacitated vehicle routing problem: A visual Benchmarking Approach for sustainable manufacturing
    2017 IEEE International Conference on Service Operations and Logistics and Informatics (SOLI), 2017
    Co-Authors: Michele Dassisti, Yasamin Eslami, Matin Mohaghegh

    Abstract:

    Optimisation problem concerning material flows, to increase the efficiency while reducing relative resource consumption is one of the most pressing problems today. The focus point of this study is to propose a new visual Benchmarking Approach to select the best material-flow path from the depot to the production lines, referring to the well-known Capacitated Vehicle Routing Problem (CVRP). An example industrial case study is considered to this aim. Two different solution techniques were adopted (namely Mixed Integer Linear Programming and the Ant Colony Optimization) in searching optimal solutions to the CVRP. The visual Benchmarking proposed, based on the persistent homology Approach, allowed to support the comparison of the optimal solutions based on the entropy of the output in different scenarios. Finally, based on the non-standard measurements of Crossing Length Percentage (CLP), the visual Benchmarking procedure makes it possible to find the most practical and applicable solution to CVRP by considering the visual attractiveness and the quality of the routes.

  • Raw material flow optimization as a capacitated vehicle routing problem: A visual Benchmarking Approach for sustainable manufacturing
    2017 IEEE International Conference on Service Operations and Logistics and Informatics (SOLI), 2017
    Co-Authors: Michele Dassisti, Yasamin Eslami, Matin Mohaghegh

    Abstract:

    Optimisation problem concerning material flows, to increase the efficiency while reducing relative resource consumption is one of the most pressing problems today. The focus point of this study is to propose a new visual Benchmarking Approach to select the best material-flow path from the depot to the production lines, referring to the well-known Capacitated Vehicle Routing Problem (CVRP). An example industrial case study is considered to this aim. Two different solution techniques were adopted (namely Mixed Integer Linear Programming and the Ant Colony Optimization) in searching optimal solutions to the CVRP. The visual Benchmarking proposed, based on the persistent homology Approach, allowed to support the comparison of the optimal solutions based on the entropy of the output in different scenarios. Finally, based on the non-standard measurements of Crossing Length Percentage (CLP), the visual Benchmarking procedure makes it possible to find the most practical and applicable solution to CVRP by considering the visual attractiveness and the quality of the routes.