The Experts below are selected from a list of 12504 Experts worldwide ranked by ideXlab platform

Madhusudan Srinivasan - One of the best experts on this subject based on the ideXlab platform.

  • prioritization of metamorphic relations based on Test Case Execution properties
    International Symposium on Software Reliability Engineering, 2018
    Co-Authors: Madhusudan Srinivasan
    Abstract:

    A Test oracle is essential for software Testing. In certain complex systems, it is hard to distinguish between correct and incorrect behavior. Metamorphic Testing is one of the solution to solve the Test oracle problem. In metamorphic Testing, metamorphic relations (MRs) are derived based on the properties exhibited by the program under Test (PUT). These MRs play a major role in the generation of Test data for conducting MT. The effectiveness of MRs can be determined based on the ability to detect considerable faults for the given PUT. Many metamorphic relations with different fault finding capability can be used to Test the PUT and it is important to identify and prioritize the MRs based on its fault finding effectiveness. In order to answer this challenge, we propose to prioritize the MRs based on the diversity in the Execution path of the source and follow-up Test Cases of the MRs. We propose four metrics to capture different levels of diversity in the Execution behavior of the Test Cases for each of the derived MRs. The total weight calculated for each of the MRs using the metrics is used to prioritize the MRs.

  • ISSRE Workshops - Prioritization of Metamorphic Relations Based on Test Case Execution Properties
    2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), 2018
    Co-Authors: Madhusudan Srinivasan
    Abstract:

    A Test oracle is essential for software Testing. In certain complex systems, it is hard to distinguish between correct and incorrect behavior. Metamorphic Testing is one of the solution to solve the Test oracle problem. In metamorphic Testing, metamorphic relations (MRs) are derived based on the properties exhibited by the program under Test (PUT). These MRs play a major role in the generation of Test data for conducting MT. The effectiveness of MRs can be determined based on the ability to detect considerable faults for the given PUT. Many metamorphic relations with different fault finding capability can be used to Test the PUT and it is important to identify and prioritize the MRs based on its fault finding effectiveness. In order to answer this challenge, we propose to prioritize the MRs based on the diversity in the Execution path of the source and follow-up Test Cases of the MRs. We propose four metrics to capture different levels of diversity in the Execution behavior of the Test Cases for each of the derived MRs. The total weight calculated for each of the MRs using the metrics is used to prioritize the MRs.

Sébastien Salva - One of the best experts on this subject based on the ideXlab platform.

  • An Advanced Approach for Choosing Security Patterns and Checking their Implementation
    2019
    Co-Authors: Sébastien Salva, Loukmen Regainia
    Abstract:

    This paper tackles the problems of generating concrete Test Cases for Testing whether an application is vulnerable to attacks, and of checking whether security solutions are correctly implemented. The approach proposed in the paper aims at guiding developers towards the implementation of secure applications, from the threat modelling stage up to the Testing one. This approach relies on a knowledge base integrating varied security data, e.g., attacks, attack steps, and security patterns that are generic and re-usable solutions to design secure applications. The first stage of the approach consists in assisting developers in the design of Attack Defense Trees expressing the attacker possibilities to compromise an application and the defenses that may be implemented. These defenses are given under the form of security pattern combinations. In the second stage, these trees are used to guide developers in the Test Case generation. After the Test Case Execution, Test verdicts show whether an application is vulnerable to the threats modelled by an ADTree. The last stage of the approach checks whether behavioural properties of security patterns hold in the application traces collected while the Test Case Execution. These properties are formalised with LTL properties, which are generated from the knowledge base. Developers do not have to write LTL properties not to be expert in formal models. We experimented the approach on 10 Web applications to evaluate its Testing effectiveness and its performance.

  • Using Data Integration for Security Testing
    2017
    Co-Authors: Sébastien Salva, Loukmen Regainia
    Abstract:

    The explosion of digitisation makes a plethora of security data publicly available for developers. These numerous (often complex) documents expose them to the difficulty of choosing the most appropriate solution for securing their applications. We propose in this paper a method based upon data acquisition and integration, which assists developers in the Threat modelling stage and in the security Test Case Execution. The method firstly helps devise Attack Defense Trees by means of a data-store. These trees show attacks, steps and defenses given under the form of security patterns, which are re-usable solutions to design more secure applications. These trees are then used for the Test Case generation. The data-store integrates Test Case stubs, which make this generation easier and developers more efficient. We evaluate our approach on 24 participants and show encouraging results on the use of data integration in software engineering.

  • ICTSS - Using Data Integration for Security Testing
    Testing Software and Systems, 2017
    Co-Authors: Sébastien Salva, Loukmen Regainia
    Abstract:

    The explosion of digitisation makes a plethora of security data publicly available for developers. These numerous (often complex) documents expose them to the difficulty of choosing the most appropriate solution for securing their applications. We propose in this paper a method based upon data acquisition and integration, which assists developers in the Threat modelling stage and in the security Test Case Execution. The method firstly helps devise Attack Defense Trees by means of a data-store. These trees show attacks, steps and defenses given under the form of security patterns, which are re-usable solutions to design more secure applications. These trees are then used for the Test Case generation. The data-store integrates Test Case stubs, which make this generation easier and developers more efficient. We evaluate our approach on 24 participants and show encouraging results on the use of data integration in software engineering.

  • APSET, an Android aPplication SEcurity Testing tool for detecting intent-based vulnerabilities
    International Journal on Software Tools for Technology Transfer, 2015
    Co-Authors: Sébastien Salva, Stassia R. Zafimiharisoa
    Abstract:

    The Android messaging system, called intent, is a mechanism that ties components together to build applications for smartphones. Intents are kinds of messages composed of actions and data, sent by a component to another component to perform several operations, e.g. launching a user interface. The intent mechanism offers a lot of flexibility for developing Android applications, but it might also be used as an entry point for security attacks. The latter can be easily sent with intents to components, that can indirectly forward attacks to other components and so on. In this context, this paper proposes APSET, a tool for Android aPplication SEcurity Testing, which aims at detecting intent-based vulnerabilities. It takes as inputs Android applications and intent-based vulnerabilities formally expressed with models called vulnerability patterns. Then, and this is the originality of our approach, class diagrams and partial specifications are automatically generated from applications with algorithms reflecting some knowledge of the Android documentation. These partial specifications avoid false positives and refine the Test result with special verdicts notifying that a component is not compliant to its specification. Furthermore, we propose a Test Case Execution framework which supports the receipt of any exception, the detection of application crashes, and provides a final XML Test report detailing the Test Case verdicts. The vulnerability detection effectiveness of APSET is evaluated with experimentations on randomly chosen Android applications of the Android Market.

Andreas Fuchs - One of the best experts on this subject based on the ideXlab platform.

  • automatic scalable parallel Test Case Execution introducing the munster distributed Test Case runner for java midstr
    ACM Symposium on Applied Computing, 2018
    Co-Authors: Vincent Von Hof, Andreas Fuchs
    Abstract:

    Software Testing is a broad research field and of great relevance to practitioners. Software Testing involves multiple consecutive Testing phases. One of these phases is the is the unit-Testing phase, during which individual requirements of the software component are checked. Only once software-units and -components have been Tested, can the Testing process continue. Trivially, if this phase requires a great amount time, the Testing process is delayed and in the worst Case, the rollout of the system has to be rescheduled. Focusing on the widely adopted JUnit Test framework, we propose a system to execute Test Cases on multiple distributed virtual machines. This paper presents an approach to connect multiple Java Virtual Machines (JVMs) into a Test-net, which can execute a JUnit Test suite in parallel. Furthermore, different distribution schemata are discussed. By distributing the Execution to multiple VMs, the overall time to Test is reduced, in order to improve the overall time required for the Testing process.

Mats Carlsson - One of the best experts on this subject based on the ideXlab platform.

  • time aware Test Case Execution scheduling for cyber physical systems
    arXiv: Software Engineering, 2019
    Co-Authors: Morten Mossige, Arnaud Gotlieb, Helge Spieker, Hein Meling, Mats Carlsson
    Abstract:

    Testing cyber-physical systems involves the Execution of Test Cases on target-machines equipped with the laTest release of a software control system. When Testing industrial robots, it is common that the target machines need to share some common resources, e.g., costly hardware devices, and so there is a need to schedule Test Case Execution on the target machines, accounting for these shared resources. With a large number of such Tests executed on a regular basis, this scheduling becomes difficult to manage manually. In fact, with manual Test Execution planning and scheduling, some robots may remain unoccupied for long periods of time and some Test Cases may not be executed. This paper introduces TC-Sched, a time-aware method for automated Test Case Execution scheduling. TC-Sched uses Constraint Programming to schedule Tests to run on multiple machines constrained by the Tests' access to shared resources, such as measurement or networking devices. The CP model is written in SICStus Prolog and uses the Cumulatives global constraint. Given a set of Test Cases, a set of machines, and a set of shared resources, TC-Sched produces an Execution schedule where each Test is executed once with minimal time between when a source code change is committed and the Test results are reported to the developer. Experiments reveal that TC-Sched can schedule 500 Test Cases over 100 machines in less than 4 minutes for 99.5% of the instances. In addition, TC-Sched largely outperforms simpler methods based on a greedy algorithm and is suitable for deployment on industrial robot Testing.

  • CP - Time-Aware Test Case Execution Scheduling for Cyber-Physical Systems
    Lecture Notes in Computer Science, 2017
    Co-Authors: Morten Mossige, Arnaud Gotlieb, Helge Spieker, Hein Meling, Mats Carlsson
    Abstract:

    Testing cyber-physical systems involves the Execution of Test Cases on target-machines equipped with the laTest release of a software control system. When Testing industrial robots, it is common that the target machines need to share some common resources, e.g., costly hardware devices, and so there is a need to schedule Test Case Execution on the target machines, accounting for these shared resources. With a large number of such Tests executed on a regular basis, this scheduling becomes difficult to manage manually. In fact, with manual Test Execution planning and scheduling, some robots may remain unoccupied for long periods of time and some Test Cases may not be executed.

Marius Liaaen - One of the best experts on this subject based on the ideXlab platform.

  • ICST - REMAP: Using Rule Mining and Multi-objective Search for Dynamic Test Case Prioritization
    2018 IEEE 11th International Conference on Software Testing Verification and Validation (ICST), 2018
    Co-Authors: Dipesh Pradhan, Shuai Wang, Shaukat Ali, Tao Yue, Marius Liaaen
    Abstract:

    Test Case prioritization (TP) prioritizes Test Cases into an optimal order for achieving specific criteria (e.g., higher fault detection capability) as early as possible. However, the existing TP techniques usually only produce a static Test Case order before the Execution without taking runtime Test Case Execution results into account. In this paper, we propose an approach for black-box dynamic TP using rule mining and multi-objective search (named as REMAP). REMAP has three key components: 1) Rule Miner, which mines Execution relations among Test Cases from historical Execution data; 2) Static Prioritizer, which defines two objectives (i.e., fault detection capability (FDC) and Test Case reliance score (TRS)) and applies multi-objective search to prioritize Test Cases statically; and 3) Dynamic Executor and Prioritizer, which executes statically-prioritized Test Cases and dynamically updates the Test Case order based on the runtime Test Case Execution results. We empirically evaluated REMAP with random search, greedy based on FDC, greedy based on FDC and TRS, static search-based prioritization, and rule-based prioritization using two industrial and three open source Case studies. Results showed that REMAP significantly outperformed the other approaches for 96% of the Case studies and managed to achieve on average 18% higher Average Percentage of Faults Detected (APFD).

  • STIPI: Using Search to Prioritize Test Cases Based on Multi-objectives Derived from Industrial Practice
    2016
    Co-Authors: Dipesh Pradhan, Shuai Wang, Marius Liaaen
    Abstract:

    The importance of cost-effectively prioritizing Test Cases is undeniable in automated Testing practice in industry. This paper focuses on prioritizing Test Cases developed to Test product lines of Video Conferencing Systems (VCSs) at Cisco Systems, Norway. Each Test Case requires setting up configurations of a set of VCSs, invoking a set of Test APIs with specific inputs, and checking statuses of the VCSs under Test. Based on these characteristics and available information related with Test Case Execution (e.g., number of faults detected), we identified that the Test Case prioritization problem in our particular context should focus on achieving high coverage of configurations, Test APIs, statuses, and high fault detection capability as quickly as possible. To solve this problem, we propose a search-based Test Case prioritization approach (named STIPI) by defining a fitness function with four objectives and integrating it with a widely applied multi-objective optimization algorithm (named Non-dominated Sorting Genetic Algorithm II). We compared STIPI with random search (RS), Greedy algorithm, and three approaches adapted from literature, using three real sets of Test Cases from Cisco with four time budgets (25 %, 50 %, 75 % and 100 %). Results show that STIPI significantly outperformed the selected approaches and managed to achieve better performance than RS for on average 39.9 %, 18.6 %, 32.7 % and 43.9 % for the coverage of configurations, Test APIs, statuses and fault detection capability, respectively.

  • ICSE (Companion Volume) - Enhancing Test Case prioritization in an industrial setting with resource awareness and multi-objective search
    Proceedings of the 38th International Conference on Software Engineering Companion - ICSE '16, 2016
    Co-Authors: Shuai Wang, Shaukat Ali, Tao Yue, Oyvind Bakkeli, Marius Liaaen
    Abstract:

    Test Case prioritization is an essential part of Test Execution systems for large organizations developing software systems in the context that their software versions are released very frequently. They must be Tested on a variety of compatible hardware with different configurations to ensure correct functioning of a software version on a compatible hardware. In practice, Test Case Execution must not only execute cost-effective Test Cases in an optimal order, but also optimally allocate required Test resources, in order to deliver high quality software releases. To optimize the current Test Execution system for Testing software releases developed for Videoconferencing Systems (VCSs) at Cisco, Norway, in this paper, we propose a resource-aware multi-objective optimization solution with a fitness function defined based on four cost-effectiveness measures. In this context, a set of software releases must be Tested on a set of compatible VCS hardware (Test resources) by executing a set of cost-effective Test Cases in an optimal order within a given Test cycle constrained by maximum allowed time budget and maximum available Test resources. We empirically evaluated seven search algorithms regarding their performance and scalability by comparing with the current practice (random ordering (RO)). The results show that the proposed solution with the best search algorithm (i.e., Random-Weighted Genetic Algorithm) improved the current practice by reducing on average 40.6% of time for Test resource allocation and Test Case Execution, improved Test resource usage on average by 37.9% and fault detection on average by 60%.