Structural Coverage

14,000,000 Leading Edge Experts on the ideXlab platform

Scan Science and Technology

Contact Leading Edge Experts & Companies

Scan Science and Technology

Contact Leading Edge Experts & Companies

The Experts below are selected from a list of 31353 Experts worldwide ranked by ideXlab platform

Thomas Szyperski - One of the best experts on this subject based on the ideXlab platform.

Michael Grottke - One of the best experts on this subject based on the ideXlab platform.

  • a vector markov model for Structural Coverage growth and the number of failure occurrences
    International Symposium on Software Reliability Engineering, 2002
    Co-Authors: Michael Grottke
    Abstract:

    Most software reliability growth models specify the expected number of failures experienced as a function of testing effort or calendar time. However, there are approaches to model the development of intermediate factors driving failure occurrences. This paper starts out with presenting a model framework consisting of four consecutive relationships. It is shown that a differential equation representing this framework is a generalization of several finite failures category models. The relationships between the number of test cases executed and expected Structural Coverage, and between expected Structural Coverage and the expected number of failure occurrences are then explored further. A vector Markov model allowing for partial redundancy in sampling code constructs is developed. The model bridges the gap between setups related to operational testing and systematic testing, respectively. Two extensions of the partial redundancy model considering the development of the number of failure occurrences are discussed. The paper concludes with showing that the extended models fit into the structure of the differential equation presented at the beginning, which permits further interpretation.

  • Modelling Structural Coverage and the number of failure occurrences with non-homogeneous Markov chains
    2001
    Co-Authors: Michael Grottke
    Abstract:

    Most software reliability growth models specify the expected number of failures experienced as a function of testing effort or calendar time. However, there are approaches to model the development of intermediate factors driving failure occurrences. This paper starts out with presenting a model framework consisting of four consecutive relationships. It is shown that a differential equation representing this framework is a generalization of several finite failures category models. The relationships between the number of test cases executed and expected Structural Coverage, and between expected Structural Coverage and the expected number of failure occurrences are then explored further. A non-homogeneous Markov model allowing for partial redundancy in sampling code constructs is developed. The model bridges the gap between setups related to operational testing and systematic testing, respectively. Two extensions of the model considering the development of the number of failure occurrences are discussed. The paper concludes with showing that the extended models fit into the structure of the differential equation presented at the beginning, which permits further interpretation.

  • ISSRE - A vector Markov model for Structural Coverage growth and the number of failure occurrences
    13th International Symposium on Software Reliability Engineering 2002. Proceedings., 1
    Co-Authors: Michael Grottke
    Abstract:

    Most software reliability growth models specify the expected number of failures experienced as a function of testing effort or calendar time. However, there are approaches to model the development of intermediate factors driving failure occurrences. This paper starts out with presenting a model framework consisting of four consecutive relationships. It is shown that a differential equation representing this framework is a generalization of several finite failures category models. The relationships between the number of test cases executed and expected Structural Coverage, and between expected Structural Coverage and the expected number of failure occurrences are then explored further. A vector Markov model allowing for partial redundancy in sampling code constructs is developed. The model bridges the gap between setups related to operational testing and systematic testing, respectively. Two extensions of the partial redundancy model considering the development of the number of failure occurrences are discussed. The paper concludes with showing that the extended models fit into the structure of the differential equation presented at the beginning, which permits further interpretation.

Mats P E Heimdahl - One of the best experts on this subject based on the ideXlab platform.

  • the risks of Coverage directed test case generation
    IEEE Transactions on Software Engineering, 2015
    Co-Authors: Matt Staats, Michael W Whalen, Mats P E Heimdahl
    Abstract:

    A number of Structural Coverage criteria have been proposed to measure the adequacy of testing efforts. In the avionics and other critical systems domains, test suites satisfying Structural Coverage criteria are mandated by standards. With the advent of powerful automated test generation tools, it is tempting to simply generate test inputs to satisfy these Structural Coverage criteria. However, while techniques to produce Coverage-providing tests are well established, the effectiveness of such approaches in terms of fault detection ability has not been adequately studied. In this work, we evaluate the effectiveness of test suites generated to satisfy four Coverage criteria through counterexample-based test generation and a random generation approach—where tests are randomly generated until Coverage is achieved—contrasted against purely random test suites of equal size. Our results yield three key conclusions. First, Coverage criteria satisfaction alone can be a poor indication of fault finding effectiveness, with inconsistent results between the seven case examples (and random test suites of equal size often providing similar—or even higher—levels of fault finding). Second, the use of Structural Coverage as a supplement—rather than a target—for test generation can have a positive impact, with random test suites reduced to a Coverage-providing subset detecting up to 13.5 percent more faults than test suites generated specifically to achieve Coverage. Finally, Observable MC/DC, a criterion designed to account for program structure and the selection of the test oracle, can—in part—address the failings of traditional Structural Coverage criteria, allowing for the generation of test suites achieving higher levels of fault detection than random test suites of equal size. These observations point to risks inherent in the increase in test automation in critical systems, and the need for more research in how Coverage criteria, test generation approaches, the test oracle used, and system structure jointly influence test effectiveness.

  • ISSRE - Efficient observability-based test generation by dynamic symbolic execution
    2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), 2015
    Co-Authors: Dongjiang You, Michael W Whalen, Sanjai Rayadurgam, Mats P E Heimdahl
    Abstract:

    Structural Coverage metrics have been widely used to measure test suite adequacy as well as to generate test cases. In previous investigations, we have found that the fault-finding effectiveness of tests satisfying Structural Coverage criteria is highly dependent on program syntax — even if the faulty code is exercised, its effect may not be observable at the output. To address these problems, observability-based Coverage metrics have been defined. Specifically, Observable MC/DC (OMC/DC) is a criterion that appears to be both more effective at detecting faults and more robust to program restructuring than MC/DC. Traditional counterexample-based test generation for OMC/DC, however, can be infeasible on large systems. In this study, we propose an incremental test generation approach that combines the notion of observability with dynamic symbolic execution. We evaluated the efficiency and effectiveness of our approach using seven systems from the avionics and medical device domains. Our results show that the incremental approach requires much lower generation time, while achieving even higher fault finding effectiveness compared with regular OMC/DC generation.

  • SBST - Moving the goalposts: Coverage satisfaction is not enough
    Proceedings of the 7th International Workshop on Search-Based Software Testing - SBST 2014, 2014
    Co-Authors: Matt Staats, Michael W Whalen, Mats P E Heimdahl
    Abstract:

    Structural Coverage criteria have been proposed to measure the adequacy of testing efforts. Indeed, in some domains—e.g., critical systems areas—Structural Coverage criteria must be satisfied to achieve certification. The advent of powerful search-based test generation tools has given us the ability to generate test inputs to satisfy these Structural Coverage criteria. While tempting, recent empirical evidence indicates these tools should be used with caution, as merely achieving high Structural Coverage is not necessarily indicative of high fault detection ability. In this report, we review some of these findings, and offer recommendations on how the strengths of search-based test generation methods can alleviate these issues.

  • Testing Strategies for Model-Based Development
    2013
    Co-Authors: Mats P E Heimdahl, Michael W Whalen, Ajitha Rajan, Steven P. Miller
    Abstract:

    Abstract Model-based software development offers new opportunities and challenges for validation and verification of safety-critical software. Since models have well-defined syntax and semantics, it is possible to test models as well as source code and to define Structural Coverage metrics over models. Further, given a formal description of requirements, it is possible to use automated tools to check whether models satisfy requirements and to describe objective notions of requirements Coverage. Recently, model-based testing tools have emerged that allow auto-generation of test-cases given a model and Structural Coverage metrics. Nevertheless, there is a great deal of uncertainty as to which tools and techniques are effective and how to structure a testing process using model-based development. In this report, we describe some of the issues in model-based testing, present an approach for testing artifacts generated by a model-based development process, and describe the relationship between this process and existing standards such as DO-178B. This approach divides the traditional testing process into two parts:

  • on the danger of Coverage directed test case generation
    Fundamental Approaches to Software Engineering, 2012
    Co-Authors: Matt Staats, Michael W Whalen, Mats P E Heimdahl
    Abstract:

    In the avionics domain, the use of Structural Coverage criteria is legally required in determining test suite adequacy. With the success of automated test generation tools, it is tempting to use these criteria as the basis for test generation. To more firmly establish the effectiveness of such approaches, we have generated and evaluated test suites to satisfy two Coverage criteria using counterexample-based test generation and a random generation approach, contrasted against purely random test suites of equal size. Our results yield two key conclusions. First, Coverage criteria satisfaction alone is a poor indication of test suite effectiveness. Second, the use of Structural Coverage as a supplement--not a target--for test generation can have a positive impact. These observations points to the dangers inherent in the increase in test automation in critical systems and the need for more research in how Coverage criteria, generation approach, and system structure jointly influence test effectiveness.

Thomas B. Acton - One of the best experts on this subject based on the ideXlab platform.

  • Solution NMR structures of homeodomains from human proteins ALX4, ZHX1, and CASP8AP2 contribute to the Structural Coverage of the Human Cancer Protein Interaction Network
    Journal of Structural and Functional Genomics, 2014
    Co-Authors: Xianzhong Xu, Alexander Eletsky, Surya V. S. R. K. Pulavarti, Yuanpeng Janet Huang, John K. Everett, Thomas B. Acton, Gaetano T. Montelione, Rong Xiao, Thomas Szyperski
    Abstract:

    High-quality solution NMR structures of three homeodomains from human proteins ALX4, ZHX1 and CASP8AP2 were solved. These domains were chosen as targets of a biomedical theme project pursued by the Northeast Structural Genomics Consortium. This project focuses on increasing the Structural Coverage of human proteins associated with cancer.

  • Solution NMR structures of immunoglobulin-like domains 7 and 12 from obscurin-like protein 1 contribute to the Structural Coverage of the human cancer protein interaction network
    Journal of Structural and Functional Genomics, 2014
    Co-Authors: Surya V. S. R. K. Pulavarti, John K. Everett, Thomas B. Acton, Gaetano T. Montelione, Rong Xiao, Yuanpeng J. Huang, Kari Pederson, James H. Prestegard, Thomas Szyperski
    Abstract:

    High-quality solution NMR structures of immunoglobulin-like domains 7 and 12 from human obscurin-like protein 1 were solved. The two domains share 30 % sequence identity and their structures are, as expected, rather similar. The new structures contribute to Structural Coverage of human cancer associated proteins. Mutations of Arg 812 in domain 7 cause the rare 3-M syndrome, and this site is located in a surface area predicted to be involved in proteinprotein interactions.

  • Solution NMR structures provide first Structural Coverage of the large protein domain family PF08369 and complementary Structural Coverage of dark operative protochlorophyllide oxidoreductase complexes
    Journal of Structural and Functional Genomics, 2013
    Co-Authors: Surya V. S. R. K. Pulavarti, Alexander Eletsky, John K. Everett, Thomas B. Acton, Gaetano T. Montelione, Rong Xiao, Erik A. Feldmann, Michael A. Kennedy, Thomas Szyperski
    Abstract:

    High-quality NMR structures of the C-terminal domain comprising residues 484–537 of the 537-residue protein Bacterial chlorophyll subunit B (BchB) from Chlorobium tepidum and residues 9–61 of 61-residue Asr4154 from Nostoc sp. (strain PCC 7120) exhibit a mixed α/β fold comprised of three α-helices and a small β-sheet packed against second α-helix. These two proteins share 29 % sequence similarity and their structures are globally quite similar. The structures of BchB(484–537) and Asr4154(9–61) are the first representative structures for the large protein family (Pfam) PF08369, a family of unknown function currently containing 610 members in bacteria and eukaryotes. Furthermore, BchB(484–537) complements the Structural Coverage of the dark-operating protochlorophyllide oxidoreductase.

  • Structural genomics is the largest contributor of novel Structural leverage
    Journal of Structural and Functional Genomics, 2009
    Co-Authors: Rajesh Nair, John K. Everett, Thomas B. Acton, Andras Fiser, Lukasz Jaroszewski, Adam Godzik, Ta Tsen Soong, Andrei Kouranov, Christine A. Orengo
    Abstract:

    The Protein Structural Initiative (PSI) at the US National Institutes of Health (NIH) is funding four large-scale centers for Structural genomics (SG). These centers systematically target many large families without Structural Coverage, as well as very large families with inadequate Structural Coverage. Here, we report a few simple metrics that demonstrate how successfully these efforts optimize Structural Coverage: while the PSI-2 (2005-now) contributed more than 8% of all structures deposited into the PDB, it contributed over 20% of all novel structures (i.e. structures for protein sequences with no Structural representative in the PDB on the date of deposition). The Structural Coverage of the protein universe represented by today’s UniProt (v12.8) has increased linearly from 1992 to 2008; Structural genomics has contributed significantly to the maintenance of this growth rate. Success in increasing novel leverage (defined in Liu et al. in Nat Biotechnol 25:849–851, 2007) has resulted from systematic targeting of large families. PSI’s per structure contribution to novel leverage was over 4-fold higher than that for non-PSI Structural biology efforts during the past 8 years. If the success of the PSI continues, it may just take another ~15 years to cover most sequences in the current UniProt database.

  • Solution NMR structure of Escherichia coli ytfP expands the Structural Coverage of the UPF0131 protein domain family.
    Proteins: Structure Function and Bioinformatics, 2007
    Co-Authors: James M. Aramini, Thomas B. Acton, Rong Xiao, Yuanpeng J. Huang, G. V. T. Swapna, John R. Cort, P. K. Rajan, R. Shastry, Jinfeng Liu, Burkhard Rost
    Abstract:

    Solution NMR structure of Escherichia coli ytfP expands the Structural Coverage of the UPF0131 protein domain family James M. Aramini,* Yuanpeng J. Huang, G.V.T. Swapna, John R. Cort, P.K. Rajan, Rong Xiao, Ritu Shastry, Thomas B. Acton, Jinfeng Liu, Burkhard Rost, Michael A. Kennedy, and Gaetano T. Montelione* 1 Center for Advanced Biotechnology and Medicine, Department of Molecular Biology and Biochemistry, Rutgers University, Piscataway, New Jersey 08854 2Northeast Structural Genomics Consortium, Rutgers University, Piscataway, New Jersey 08854 3 Biological Sciences Division, Pacific Northwest National Laboratory, Richland, Washington 99352 4Department of Biochemistry and Molecular Biophysics, Columbia University, New York, New York 10032 5Department of Biochemistry, Robert Wood Johnson Medical School, University of Medicine and Dentistry of New Jersey, Piscataway, New Jersey 08854

Goiuria Sagardui - One of the best experts on this subject based on the ideXlab platform.

  • test case selection using Structural Coverage in software product lines for time budget constrained scenarios
    ACM Symposium on Applied Computing, 2019
    Co-Authors: Urtzi Markiegi, Aitor Arrieta, Leire Etxeberria, Goiuria Sagardui
    Abstract:

    Testing product lines is a challenging activity due to the large number of products to be tested. Many approaches focus on reducing the time for testing a product line by reducing the number of products to be tested, by employing, for instance, combinatorial approaches. However, even if the number of derived products by a combinatorial approach is limited, testing can still be time consuming. In this paper, we propose three different test case selection methods that consider a given time budget to test product lines in an efficient manner using Structural Coverage information. We analyze the three methods with three white-box Coverage criteria (i.e., Decision Coverage, Condition Coverage and Modified Condition/Decision Coverage). We evaluate the different approaches with a case study from the automotive domain and mutation testing. The results suggest that considering Coverage information at the domain engineering level helps on detecting more faults, particularly when time budgets are low.

  • SAC - Test case selection using Structural Coverage in software product lines for time-budget constrained scenarios
    Proceedings of the 34th ACM SIGAPP Symposium on Applied Computing, 2019
    Co-Authors: Urtzi Markiegi, Aitor Arrieta, Leire Etxeberria, Goiuria Sagardui
    Abstract:

    Testing product lines is a challenging activity due to the large number of products to be tested. Many approaches focus on reducing the time for testing a product line by reducing the number of products to be tested, by employing, for instance, combinatorial approaches. However, even if the number of derived products by a combinatorial approach is limited, testing can still be time consuming. In this paper, we propose three different test case selection methods that consider a given time budget to test product lines in an efficient manner using Structural Coverage information. We analyze the three methods with three white-box Coverage criteria (i.e., Decision Coverage, Condition Coverage and Modified Condition/Decision Coverage). We evaluate the different approaches with a case study from the automotive domain and mutation testing. The results suggest that considering Coverage information at the domain engineering level helps on detecting more faults, particularly when time budgets are low.