The Experts below are selected from a list of 76413 Experts worldwide ranked by ideXlab platform
Jan Bosch - One of the best experts on this subject based on the ideXlab platform.
-
Test activities in the continuous integration and delivery pipeline
Journal of Software: Evolution and Process, 2019Co-Authors: Torvald Martensson, Daniel Ståhl, Jan BoschAbstract:Based on 25 interviews with participants from four case study companies that develop large-scale software embedded systems, this paper presents the Test Activity Stakeholders (TAS) model. The TAS model shows how the continuous integration and delivery pipeline can be designed to include test activities that support four stakeholder interests: “Check changes,” “Secure stability,” “Measure progress,” and “Verify compliance.” The model is developed to show how each of the stakeholder interests are best supported by unit/component tests or system tests, by automated testing or manual testing and by tests executed in simulated environments or on real hardware. The TAS model may serve as a starting point for companies when evaluating and designing their continuous integration and delivery pipeline. The validation of the TAS model included twelve individuals from three case study companies. The validation showed that the TAS model is actionable and useful in practice, enabling the identification of stakeholders and showing where improvement efforts should be focused in order to support all stakeholder interests in the continuous integration and delivery pipeline.
-
ICSSP - Cinders: the continuous integration and delivery architecture framework
Proceedings of the 2018 International Conference on Software and System Process, 2018Co-Authors: Daniel Ståhl, Jan BoschAbstract:This extended abstract summarizes an article, which has been published in Information and Software Technology and was selected for the Journal-First presentations at the International Conference on Software and System Process (ICSSP 2018). Full Article Reference. Daniel Stahl and Jan Bosch. 2017. Cinders: The continuous integration and delivery architecture framework. Information and Software Technology 83 (2017), 76--93.
-
The continuity of continuous integration
Journal of Systems and Software, 2017Co-Authors: Daniel Sthl, Torvald Mrtensson, Jan BoschAbstract:Metrics of the continuity of continuous integration proposed and discussed.Organizational size found to correlate with CI continuity.Evidence of correlation between continuity and organization composition discovered.Findings discussed in the context of architectural styles and manufacturability. The practice of continuous integration has firmly established itself in the mainstream of the software engineering industry, yet many questions surrounding it remain unanswered. Prominent among these is the issue of scalability: continuous integration has been reported to be possible to scale, but with difficulties. Understanding of the underlying mechanisms causing these difficulties is shallow, however: what is it about size that is problematic, which kind of size, and what aspect of continuous integration does it impede? Based on quantitative data from six industry cases encompassing close to 2000 engineers, complemented by interviews with engineers from five companies, this paper investigates the correlation between the continuity of continuous integration and size. It is found that not only is there indeed a correlation between the size and composition of a development organization and its tendency to integrate continuously; there is evidence that the size of the organization influences ways of working, which in turn correlate with the degree of continuity, raising the question of software manufacturability. It is further observed that developer behavior in ostensibly continuously integrating cases does not necessarily match expectations, and that frequent integration of the product itself does not automatically imply that each individual developer commits frequently.
-
Cinders: The continuous integration and delivery architecture framework
Information and Software Technology, 2017Co-Authors: Daniel Ståhl, Jan BoschAbstract:Context: The popular agile practices of continuous integration and delivery have become an essential part of the software development process in many companies, yet effective methods and tools to support design, description and communication of continuous integration and delivery systems are lacking. Objective: The work reported on in this paper addresses that lack by presenting Cinders — an architecture framework designed specifically to meet the needs of such systems, influenced both by prominent enterprise and software architecture frameworks as well as experiences from continuous integration and delivery modeling in industry. Method: The state of the art for systematic design and description of continuous integration and delivery systems is established through review of literature, whereupon a proposal for an architecture framework addressing requirements derived from continuous integration and delivery modeling experiences is proposed. This framework is subsequently evaluated through interviews and workshops with engineers in varying roles in three independent companies. Results: Cinders, an architecture framework designed specifically for the purpose of describing continuous integration and delivery systems is proposed and confirmed to constitute an improvement over previous methods. This work presents software professionals with a demonstrably effective method for describing their continuous integration and delivery systems from multiple points of view and supporting multiple use-cases, including system design, communication and documentation. Conclusion: It is concluded that an architecture framework for the continuous integration and delivery domain has value; at the same time potential for further improvement is identified, particularly in the area of tool support for data collection as well as for manual modeling.
-
ICSA - continuous integration Impediments in Large-Scale Industry Projects
2017 IEEE International Conference on Software Architecture (ICSA), 2017Co-Authors: Torvald Martensson, Daniel Ståhl, Jan BoschAbstract:Based on interviews with 20 developers from two case study companies that develop large-scale software-intensive embedded systems, this paper presents the main factors that affect how often developers deliver software to the mainline. Further on, the paper describes the continuous integration behaviors in projects where up to 1,000 developers commit to the same mainline. The main factors that could enable more frequent integration of software are: "Activity planning and execution", "System thinking", "Speed" and "Confidence through test activities". Behind these main themes we also present a wide range of sub-categories ("Modular and loosely coupled architecture", "Test selection" etc) which summarizes what the developers themselves see as the continuous integration impediments in large-scale industry projects.
Daniel Ståhl - One of the best experts on this subject based on the ideXlab platform.
-
Test activities in the continuous integration and delivery pipeline
Journal of Software: Evolution and Process, 2019Co-Authors: Torvald Martensson, Daniel Ståhl, Jan BoschAbstract:Based on 25 interviews with participants from four case study companies that develop large-scale software embedded systems, this paper presents the Test Activity Stakeholders (TAS) model. The TAS model shows how the continuous integration and delivery pipeline can be designed to include test activities that support four stakeholder interests: “Check changes,” “Secure stability,” “Measure progress,” and “Verify compliance.” The model is developed to show how each of the stakeholder interests are best supported by unit/component tests or system tests, by automated testing or manual testing and by tests executed in simulated environments or on real hardware. The TAS model may serve as a starting point for companies when evaluating and designing their continuous integration and delivery pipeline. The validation of the TAS model included twelve individuals from three case study companies. The validation showed that the TAS model is actionable and useful in practice, enabling the identification of stakeholders and showing where improvement efforts should be focused in order to support all stakeholder interests in the continuous integration and delivery pipeline.
-
ICSSP - Cinders: the continuous integration and delivery architecture framework
Proceedings of the 2018 International Conference on Software and System Process, 2018Co-Authors: Daniel Ståhl, Jan BoschAbstract:This extended abstract summarizes an article, which has been published in Information and Software Technology and was selected for the Journal-First presentations at the International Conference on Software and System Process (ICSSP 2018). Full Article Reference. Daniel Stahl and Jan Bosch. 2017. Cinders: The continuous integration and delivery architecture framework. Information and Software Technology 83 (2017), 76--93.
-
Cinders: The continuous integration and delivery architecture framework
Information and Software Technology, 2017Co-Authors: Daniel Ståhl, Jan BoschAbstract:Context: The popular agile practices of continuous integration and delivery have become an essential part of the software development process in many companies, yet effective methods and tools to support design, description and communication of continuous integration and delivery systems are lacking. Objective: The work reported on in this paper addresses that lack by presenting Cinders — an architecture framework designed specifically to meet the needs of such systems, influenced both by prominent enterprise and software architecture frameworks as well as experiences from continuous integration and delivery modeling in industry. Method: The state of the art for systematic design and description of continuous integration and delivery systems is established through review of literature, whereupon a proposal for an architecture framework addressing requirements derived from continuous integration and delivery modeling experiences is proposed. This framework is subsequently evaluated through interviews and workshops with engineers in varying roles in three independent companies. Results: Cinders, an architecture framework designed specifically for the purpose of describing continuous integration and delivery systems is proposed and confirmed to constitute an improvement over previous methods. This work presents software professionals with a demonstrably effective method for describing their continuous integration and delivery systems from multiple points of view and supporting multiple use-cases, including system design, communication and documentation. Conclusion: It is concluded that an architecture framework for the continuous integration and delivery domain has value; at the same time potential for further improvement is identified, particularly in the area of tool support for data collection as well as for manual modeling.
-
Large scale continuous integration and delivery: Making great software better and faster
2017Co-Authors: Daniel StåhlAbstract:Since the inception of continuous integration, and later continuous delivery, the methods of producing software in the industry have changed dramatically over the last two decades. Automated, rapid and frequent compilation, integration, testing, analysis, packaging and delivery of new software versions have become commonplace. This change has had significant impact not only on software engineering practice, but on the way we as consumers and indeed as a society relate to software. Moreover, as we live in an increasingly software-intensive and software-dependent world, the quality and reliability of the systems we use to build, test and deliver that software is a crucial concern. At the same time, it is repeatedly shown that the successful and effective implementation of continuous engineering practices is far from trivial, particularly in a large scale context. This thesis approaches the software engineering practices of continuous integration and delivery from multiple points of view, and is split into three parts, accordingly. Part I focuses on understanding the nature of continuous integration and differences in its interpretation and implementation. In order to address this divergence and provide practitioners and researchers alike with better and less ambiguous methods for describing and designing continuous integration and delivery systems, Part II applies the paradigm of system modeling to continuous integration and delivery. Meanwhile, Part III addresses the problem of traceability. Unique challenges to traceability in the context of continuous practices are highlighted, and possible solutions are presented and evaluated.
-
ICSA - continuous integration Impediments in Large-Scale Industry Projects
2017 IEEE International Conference on Software Architecture (ICSA), 2017Co-Authors: Torvald Martensson, Daniel Ståhl, Jan BoschAbstract:Based on interviews with 20 developers from two case study companies that develop large-scale software-intensive embedded systems, this paper presents the main factors that affect how often developers deliver software to the mainline. Further on, the paper describes the continuous integration behaviors in projects where up to 1,000 developers commit to the same mainline. The main factors that could enable more frequent integration of software are: "Activity planning and execution", "System thinking", "Speed" and "Confidence through test activities". Behind these main themes we also present a wide range of sub-categories ("Modular and loosely coupled architecture", "Test selection" etc) which summarizes what the developers themselves see as the continuous integration impediments in large-scale industry projects.
John Penix - One of the best experts on this subject based on the ideXlab platform.
-
techniques for improving regression testing in continuous integration development environments
Foundations of Software Engineering, 2014Co-Authors: Sebastian Elbaum, Gregg Rothermel, John PenixAbstract:In continuous integration development environments, software engineers frequently integrate new or changed code with the mainline codebase. This can reduce the amount of code rework that is needed as systems evolve and speed up development time. While continuous integration processes traditionally require that extensive testing be performed following the actual submission of code to the codebase, it is also important to ensure that enough testing is performed prior to code submission to avoid breaking builds and delaying the fast feedback that makes continuous integration desirable. In this work, we present algorithms that make continuous integration processes more cost-effective. In an initial pre-submit phase of testing, developers specify modules to be tested, and we use regression test selection techniques to select a subset of the test suites for those modules that render that phase more cost-effective. In a subsequent post-submit phase of testing, where dependent modules as well as changed modules are tested, we use test case prioritization techniques to ensure that failures are reported more quickly. In both cases, the techniques we utilize are novel, involving algorithms that are relatively inexpensive and do not rely on code coverage information -- two requirements for conducting testing cost-effectively in this context. To evaluate our approach, we conducted an empirical study on a large data set from Google that we make publicly available. The results of our study show that our selection and prioritization techniques can each lead to cost-effectiveness improvements in the continuous integration process.
-
SIGSOFT FSE - Techniques for improving regression testing in continuous integration development environments
Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering - FSE 2014, 2014Co-Authors: Sebastian Elbaum, Gregg Rothermel, John PenixAbstract:In continuous integration development environments, software engineers frequently integrate new or changed code with the mainline codebase. This can reduce the amount of code rework that is needed as systems evolve and speed up development time. While continuous integration processes traditionally require that extensive testing be performed following the actual submission of code to the codebase, it is also important to ensure that enough testing is performed prior to code submission to avoid breaking builds and delaying the fast feedback that makes continuous integration desirable. In this work, we present algorithms that make continuous integration processes more cost-effective. In an initial pre-submit phase of testing, developers specify modules to be tested, and we use regression test selection techniques to select a subset of the test suites for those modules that render that phase more cost-effective. In a subsequent post-submit phase of testing, where dependent modules as well as changed modules are tested, we use test case prioritization techniques to ensure that failures are reported more quickly. In both cases, the techniques we utilize are novel, involving algorithms that are relatively inexpensive and do not rely on code coverage information -- two requirements for conducting testing cost-effectively in this context. To evaluate our approach, we conducted an empirical study on a large data set from Google that we make publicly available. The results of our study show that our selection and prioritization techniques can each lead to cost-effectiveness improvements in the continuous integration process.
Sebastian Elbaum - One of the best experts on this subject based on the ideXlab platform.
-
techniques for improving regression testing in continuous integration development environments
Foundations of Software Engineering, 2014Co-Authors: Sebastian Elbaum, Gregg Rothermel, John PenixAbstract:In continuous integration development environments, software engineers frequently integrate new or changed code with the mainline codebase. This can reduce the amount of code rework that is needed as systems evolve and speed up development time. While continuous integration processes traditionally require that extensive testing be performed following the actual submission of code to the codebase, it is also important to ensure that enough testing is performed prior to code submission to avoid breaking builds and delaying the fast feedback that makes continuous integration desirable. In this work, we present algorithms that make continuous integration processes more cost-effective. In an initial pre-submit phase of testing, developers specify modules to be tested, and we use regression test selection techniques to select a subset of the test suites for those modules that render that phase more cost-effective. In a subsequent post-submit phase of testing, where dependent modules as well as changed modules are tested, we use test case prioritization techniques to ensure that failures are reported more quickly. In both cases, the techniques we utilize are novel, involving algorithms that are relatively inexpensive and do not rely on code coverage information -- two requirements for conducting testing cost-effectively in this context. To evaluate our approach, we conducted an empirical study on a large data set from Google that we make publicly available. The results of our study show that our selection and prioritization techniques can each lead to cost-effectiveness improvements in the continuous integration process.
-
SIGSOFT FSE - Techniques for improving regression testing in continuous integration development environments
Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering - FSE 2014, 2014Co-Authors: Sebastian Elbaum, Gregg Rothermel, John PenixAbstract:In continuous integration development environments, software engineers frequently integrate new or changed code with the mainline codebase. This can reduce the amount of code rework that is needed as systems evolve and speed up development time. While continuous integration processes traditionally require that extensive testing be performed following the actual submission of code to the codebase, it is also important to ensure that enough testing is performed prior to code submission to avoid breaking builds and delaying the fast feedback that makes continuous integration desirable. In this work, we present algorithms that make continuous integration processes more cost-effective. In an initial pre-submit phase of testing, developers specify modules to be tested, and we use regression test selection techniques to select a subset of the test suites for those modules that render that phase more cost-effective. In a subsequent post-submit phase of testing, where dependent modules as well as changed modules are tested, we use test case prioritization techniques to ensure that failures are reported more quickly. In both cases, the techniques we utilize are novel, involving algorithms that are relatively inexpensive and do not rely on code coverage information -- two requirements for conducting testing cost-effectively in this context. To evaluate our approach, we conducted an empirical study on a large data set from Google that we make publicly available. The results of our study show that our selection and prioritization techniques can each lead to cost-effectiveness improvements in the continuous integration process.
Peter Gorm Larsen - One of the best experts on this subject based on the ideXlab platform.
-
Enabling continuous integration in a formal methods setting
International Journal on Software Tools for Technology Transfer, 2019Co-Authors: Luis Diogo Couto, Peter W. V. Tran-jørgensen, René S. Nilsson, Peter Gorm LarsenAbstract:In modern software development, the practices of continuous integration and DevOps are widely used to increase delivery speed and reduce the time it takes to deploy software changes to production. If formal method tools cannot be efficiently integrated in a DevOps paradigm, then their impact on software development will be reduced. In this paper, we present work addressing this issue through a series of extensions for the Overture tool supporting the Vienna Development Method. These extensions enable Overture to be used in a DevOps setting, through continuous integration and validation of models and generated code via integration with the Jenkins automation server. We frame the integration of formal methods and DevOps in a series of principles, demonstrate the value of this integration through a case study, and reflect on our experiences using formal methods and DevOps in an industrial setting. We hope that this work can help other formal method practitioners integrate their tools with DevOps.