application under test

14,000,000 Leading Edge Experts on the ideXlab platform

Scan Science and Technology

Contact Leading Edge Experts & Companies

Scan Science and Technology

Contact Leading Edge Experts & Companies

The Experts below are selected from a list of 94845 Experts worldwide ranked by ideXlab platform

Paolo Tonella - One of the best experts on this subject based on the ideXlab platform.

  • APOGEN: automatic page object generator for web testing
    Software Quality Journal, 2017
    Co-Authors: Andrea Stocco, Maurizio Leotta, Filippo Ricca, Paolo Tonella
    Abstract:

    Modern web applications are characterized by ultra-rapid development cycles, and web testers tend to pay scant attention to the quality of their automated end-to-end test suites. Indeed, these quickly become hard to maintain, as the application under test evolves. As a result, end-to-end automated test suites are abandoned, despite their great potential for catching regressions. The use of the Page Object pattern has proven to be very effective in end-to-end web testing. Page objects are façade classes abstracting the internals of web pages into high-level business functions that can be invoked by the test cases. By decoupling test code from web page details, web test cases are more readable and maintainable. However, the manual development of such page objects requires substantial coding effort, which is paid off only later, during software evolution. In this paper, we describe a novel approach for the automatic generation of page objects for web applications. Our approach is implemented in the tool Apogen , which automatically derives a testing model by reverse engineering the target web application. It combines clustering and static analysis to identify meaningful page abstractions that are automatically turned into Java page objects for Selenium WebDriver. Our evaluation on an open-source web application shows that our approach is highly promising: Automatically generated page object methods cover most of the application functionalities and result in readable and meaningful code, which can be very useful to support the creation of more maintainable web test suites.

  • Robula+: an algorithm for generating robust XPath locators for web testing
    Journal of Software: Evolution and Process, 2016
    Co-Authors: Maurizio Leotta, Filippo Ricca, Andrea Stocco, Paolo Tonella
    Abstract:

    Automated test scripts are used with success in many web development projects, so as to automatically verify key functionalities of the web application under test, reveal possible regressions and run a large number of tests in short time. However, the adoption of automated web testing brings advantages but also novel problems, among which the test code fragility problem. During the evolution of the web application, existing test code may easily break and testers have to correct it. In the context of automated DOM-based web testing, one of the major costs for evolving the test code is the manual effort necessary to repair broken web page element locators - lines of source code identifying the web elements e.g. form fields and buttons to interact with.

  • ICWE - Visual vs. DOM-Based Web Locators: An Empirical Study
    Lecture Notes in Computer Science, 2014
    Co-Authors: Maurizio Leotta, Diego Clerissi, Filippo Ricca, Paolo Tonella
    Abstract:

    Automation in Web testing has been successfully supported by DOM-based tools that allow testers to program the interactions of their test cases with the Web application under test. More recently a new generation of visual tools has been proposed where a test case interacts with the Web application by recognising the images of the widgets that can be actioned upon and by asserting the expected visual appearance of the result.

  • ICST - Crawlability Metrics for Web applications
    2012 IEEE Fifth International Conference on Software Testing Verification and Validation, 2012
    Co-Authors: Nadia Alshahwan, Roberto Tiella, Alessandro Marchetto, Mark Harman, Paolo Tonella
    Abstract:

    Automated web crawlers can be used to explore and exercise portions of a web application under test. However, the possibility to achieve full exploration of a web application through automated crawling is severely limited by the choice of the input values submitted with forms. Depending on the crawler's capabilities, a larger or smaller portion of web application will be automatically explored. In this paper, we introduce web crawl ability metrics to quantify properties of application pages and forms that affect crawl ability. Moreover, we show that our metrics can be used to identify the boundaries between those parts of the application that can be successfully crawled automatically and those parts that will require manual intervention or other crawl ability support. We have validated our crawl ability metrics on real web applications, for which low crawl ability was indeed associated with the existence of pages never exercised during automated crawling.

  • Crawlability metrics for automated web testing
    International Journal on Software Tools for Technology Transfer, 2011
    Co-Authors: Alessandro Marchetto, Paolo Tonella, Roberto Tiella, Nadia Alshahwan, Mark Harman
    Abstract:

    Web applications are exposed to frequent changes both in requirements and involved technologies. At the same time, there is a continuously growing demand for quality and trust and such a fast evolution and quality constraints claim for mechanisms and techniques for automated testing. Web application automated testing often involves random crawlers to navigate the application under test and automatically explore its structure. However, owing to the specific challenges of the modern Web systems, automatic crawlers may leave large portions of the application unexplored. In this paper, we propose the use of structural metrics to predict whether an automatic crawler with given crawling capabilities will be sufficient or not to achieve high coverage of the application under test. In this work, we define a taxonomy of such capabilities and we determine which combination of them is expected to give the highest reward in terms of coverage increase. Our proposal is supported by an experiment in which 19 web applications have been analyzed.

Ana C. R. Paiva - One of the best experts on this subject based on the ideXlab platform.

  • test case generation based on mutations over user execution traces
    Software Quality Journal, 2020
    Co-Authors: Ana C. R. Paiva, André Restivo, Sérgio Almeida
    Abstract:

    Automatic test case generation is usually based on models of the software under test. However, those models may not exist or may be outdated and so, the test case generation must resort to other artifacts. In a software maintenance context, test cases must adapt to software changes and should be improved continuously to test adequately the new versions of the software. Mutation testing is a fault-based testing technique that evaluates the quality of the tests by applying simple changes to the source code and checking afterwards if the tests are able to detects those changes. This paper presents a web testing approach in which test cases are generated from user execution traces as a way to deal with the absence of models. In addition, it applies mutation operators over those test cases to enrich the test suite. The mutation operators were designed so as to mimic possible real failures. The additional tests are analyzed, and those that generate different outcomes are kept because they exercise additional behavior of the web application under test. At the end, the overall approach is illustrated and validated in a case study.

  • Mobile GUI testing
    Software Quality Journal, 2018
    Co-Authors: Inês Coimbra Morgado, Ana C. R. Paiva
    Abstract:

    This paper presents a tool (iMPAcT) that automates testing of mobile applications based on the presence of recurring behaviour, UI Patterns. It combines reverse engineering, pattern matching and testing. The reverse engineering process is responsible for crawling the application, i.e. analysing the state of the application and interacting with it by firing events. The pattern matching tries to identify the presence of UI patterns based on a catalogue of patterns. When a UI Pattern from the catalogue is detected, a test strategy is applied (testing). These test strategies are called UI test Patterns. These three phases work in an iterative way: the patterns are identified and tested between firing of events, i.e. the process alternates between exploring the application and testing the UI Patterns. The process is dynamic and fully automatic not requiring any previous knowledge about the application under test. This paper presents the results of an experiment studying the reliability of the results obtained by iMPAcT. The experiment involved 25 applications found on Google Play Store and concludes that iMPAcT is successful in identifying failures in the tested patterns and that the degree of certainty of an identified failure being an actual failure is high.

  • ASE Workshops - testing Approach for Mobile applications through Reverse Engineering of UI Patterns
    2015 30th IEEE ACM International Conference on Automated Software Engineering Workshop (ASEW), 2015
    Co-Authors: Inês Coimbra Morgado, Ana C. R. Paiva
    Abstract:

    It is increasingly important to assess and ensure the correct behaviour of mobile applications as their importance in everyday life keeps increasing. This paper presents an automatic testing approach combining reverse engineering with testing. The algorithm tries to identify existing User Interface (UI) patterns on a mobile application under test through a reverse engineering process and then tests them using generic test strategies called test Patterns. The overall testing approach was implemented in the iMPAcT (Mobile PAttern testing) tool and is illustrated in a case study performed over some mobile applications as a proof-of-concept.

  • ICFEM - A model-to-implementation mapping tool for automated model-based GUI testing
    Formal Methods and Software Engineering, 2005
    Co-Authors: Ana C. R. Paiva, Joao Pascoal Faria, Nikolai Tillmann, Raul Moreira Vidal
    Abstract:

    This paper presents extensions to Spec Explorer to automate the testing of software applications through their GUIs based on a formal specification in Spec$\sharp$. Spec Explorer, a tool developed at Microsoft Research, already supports automatic generation and execution of test cases for API testing, but requires that the actions described in the model are bound to methods in a .Net assembly. The tool described in this paper extends Spec Explorer to automate GUI testing: it adds the capability to gather information about the physical GUI objects that are the target of the user actions described in the model; and it automatically generates a .Net assembly with methods that simulate those actions upon the GUI application under test. The GUI modelling and the overall test process supported by these tools are described. The approach is illustrated with the Notepad application.

  • a model to implementation mapping tool for automated model based gui testing
    Lecture Notes in Computer Science, 2005
    Co-Authors: Ana C. R. Paiva, Joao Pascoal Faria, Nikolai Tillmann, Raul Moreira Vidal
    Abstract:

    This paper presents extensions to Spec Explorer to automate the testing of software applications through their GUIs based on a formal specification in Spec#. Spec Explorer, a tool developed at Microsoft Research, already supports automatic generation and execution of test cases for API testing, but requires that the actions described in the model are bound to methods in a.Net assembly. The tool described in this paper extends Spec Explorer to automate GUI testing: it adds the capability to gather information about the physical GUI objects that are the target of the user actions described in the model; and it automatically generates a.Net assembly with methods that simulate those actions upon the GUI application under test. The GUI modelling and the overall test process supported by these tools are described. The approach is illustrated with the Notepad application.

Andrea Stocco - One of the best experts on this subject based on the ideXlab platform.

  • ESEC/SIGSOFT FSE - Vista: web test repair using computer vision
    Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering - , 2018
    Co-Authors: Andrea Stocco, Rahulkrishna Yandrapally, Ali Mesbah
    Abstract:

    Repairing broken web element locators represents the major main- tenance cost of web test cases. To detect possible repairs, testers typically inspect the tests’ interactions with the application under test through the GUI. Existing automated test repair techniques focus instead on the code and ignore visual aspects of the applica- tion. In this demo paper, we give an overview of Vista, a novel test repair technique that leverages computer vision and local crawling to automatically suggest and apply repairs to broken web tests. URL: https://github.com/saltlab/Vista

  • APOGEN: automatic page object generator for web testing
    Software Quality Journal, 2017
    Co-Authors: Andrea Stocco, Maurizio Leotta, Filippo Ricca, Paolo Tonella
    Abstract:

    Modern web applications are characterized by ultra-rapid development cycles, and web testers tend to pay scant attention to the quality of their automated end-to-end test suites. Indeed, these quickly become hard to maintain, as the application under test evolves. As a result, end-to-end automated test suites are abandoned, despite their great potential for catching regressions. The use of the Page Object pattern has proven to be very effective in end-to-end web testing. Page objects are façade classes abstracting the internals of web pages into high-level business functions that can be invoked by the test cases. By decoupling test code from web page details, web test cases are more readable and maintainable. However, the manual development of such page objects requires substantial coding effort, which is paid off only later, during software evolution. In this paper, we describe a novel approach for the automatic generation of page objects for web applications. Our approach is implemented in the tool Apogen , which automatically derives a testing model by reverse engineering the target web application. It combines clustering and static analysis to identify meaningful page abstractions that are automatically turned into Java page objects for Selenium WebDriver. Our evaluation on an open-source web application shows that our approach is highly promising: Automatically generated page object methods cover most of the application functionalities and result in readable and meaningful code, which can be very useful to support the creation of more maintainable web test suites.

  • Robula+: an algorithm for generating robust XPath locators for web testing
    Journal of Software: Evolution and Process, 2016
    Co-Authors: Maurizio Leotta, Filippo Ricca, Andrea Stocco, Paolo Tonella
    Abstract:

    Automated test scripts are used with success in many web development projects, so as to automatically verify key functionalities of the web application under test, reveal possible regressions and run a large number of tests in short time. However, the adoption of automated web testing brings advantages but also novel problems, among which the test code fragility problem. During the evolution of the web application, existing test code may easily break and testers have to correct it. In the context of automated DOM-based web testing, one of the major costs for evolving the test code is the manual effort necessary to repair broken web page element locators - lines of source code identifying the web elements e.g. form fields and buttons to interact with.

Maurizio Leotta - One of the best experts on this subject based on the ideXlab platform.

  • APOGEN: automatic page object generator for web testing
    Software Quality Journal, 2017
    Co-Authors: Andrea Stocco, Maurizio Leotta, Filippo Ricca, Paolo Tonella
    Abstract:

    Modern web applications are characterized by ultra-rapid development cycles, and web testers tend to pay scant attention to the quality of their automated end-to-end test suites. Indeed, these quickly become hard to maintain, as the application under test evolves. As a result, end-to-end automated test suites are abandoned, despite their great potential for catching regressions. The use of the Page Object pattern has proven to be very effective in end-to-end web testing. Page objects are façade classes abstracting the internals of web pages into high-level business functions that can be invoked by the test cases. By decoupling test code from web page details, web test cases are more readable and maintainable. However, the manual development of such page objects requires substantial coding effort, which is paid off only later, during software evolution. In this paper, we describe a novel approach for the automatic generation of page objects for web applications. Our approach is implemented in the tool Apogen , which automatically derives a testing model by reverse engineering the target web application. It combines clustering and static analysis to identify meaningful page abstractions that are automatically turned into Java page objects for Selenium WebDriver. Our evaluation on an open-source web application shows that our approach is highly promising: Automatically generated page object methods cover most of the application functionalities and result in readable and meaningful code, which can be very useful to support the creation of more maintainable web test suites.

  • Robula+: an algorithm for generating robust XPath locators for web testing
    Journal of Software: Evolution and Process, 2016
    Co-Authors: Maurizio Leotta, Filippo Ricca, Andrea Stocco, Paolo Tonella
    Abstract:

    Automated test scripts are used with success in many web development projects, so as to automatically verify key functionalities of the web application under test, reveal possible regressions and run a large number of tests in short time. However, the adoption of automated web testing brings advantages but also novel problems, among which the test code fragility problem. During the evolution of the web application, existing test code may easily break and testers have to correct it. In the context of automated DOM-based web testing, one of the major costs for evolving the test code is the manual effort necessary to repair broken web page element locators - lines of source code identifying the web elements e.g. form fields and buttons to interact with.

  • ICWE - Visual vs. DOM-Based Web Locators: An Empirical Study
    Lecture Notes in Computer Science, 2014
    Co-Authors: Maurizio Leotta, Diego Clerissi, Filippo Ricca, Paolo Tonella
    Abstract:

    Automation in Web testing has been successfully supported by DOM-based tools that allow testers to program the interactions of their test cases with the Web application under test. More recently a new generation of visual tools has been proposed where a test case interacts with the Web application by recognising the images of the widgets that can be actioned upon and by asserting the expected visual appearance of the result.

  • ICST - Repairing Selenium test Cases: An Industrial Case Study about Web Page Element Localization
    2013 IEEE Sixth International Conference on Software Testing Verification and Validation, 2013
    Co-Authors: Maurizio Leotta, Diego Clerissi, Filippo Ricca, Cristiano Spadaro
    Abstract:

    This poster presents an industrial case study about test automation and test suite maintenance in the context of Web applications. The Web application under test is a Learning Content Management System (eXact learning LCMS). We analysed the costs associated with the realignment of four equivalent Selenium WebDriver test suites, implemented using the page object pattern and different methods to locate web page elements, to a subsequent release of eXact learning LCMS. In our study, the two ID-based test suites required significantly less maintenance effort than the XPath-based ones.

  • ICST Workshops - Improving test Suites Maintainability with the Page Object Pattern: An Industrial Case Study
    2013 IEEE Sixth International Conference on Software Testing Verification and Validation Workshops, 2013
    Co-Authors: Maurizio Leotta, Diego Clerissi, Filippo Ricca, Cristiano Spadaro
    Abstract:

    The page object pattern is used in the context of web testing for abstracting the application's web pages in order to reduce the coupling between test cases and application under test. This paper reports on an industrial case study in a small Italian company (eXact learning solutions S.p.A.) investigating the potential benefits of adopting the page object pattern to improve the maintainability of Selenium WebDriver test cases. After a maintenance/evolution activity performed on the application under test, we compared two equivalent test suites, one built using the page object pattern and one without it. The results of our case study indicate a strong reduction in terms of time required (by a factor of about three) and number of modified LOCs (by a factor of about eight) to repair the test suite when the page object pattern is used.

Filippo Ricca - One of the best experts on this subject based on the ideXlab platform.

  • APOGEN: automatic page object generator for web testing
    Software Quality Journal, 2017
    Co-Authors: Andrea Stocco, Maurizio Leotta, Filippo Ricca, Paolo Tonella
    Abstract:

    Modern web applications are characterized by ultra-rapid development cycles, and web testers tend to pay scant attention to the quality of their automated end-to-end test suites. Indeed, these quickly become hard to maintain, as the application under test evolves. As a result, end-to-end automated test suites are abandoned, despite their great potential for catching regressions. The use of the Page Object pattern has proven to be very effective in end-to-end web testing. Page objects are façade classes abstracting the internals of web pages into high-level business functions that can be invoked by the test cases. By decoupling test code from web page details, web test cases are more readable and maintainable. However, the manual development of such page objects requires substantial coding effort, which is paid off only later, during software evolution. In this paper, we describe a novel approach for the automatic generation of page objects for web applications. Our approach is implemented in the tool Apogen , which automatically derives a testing model by reverse engineering the target web application. It combines clustering and static analysis to identify meaningful page abstractions that are automatically turned into Java page objects for Selenium WebDriver. Our evaluation on an open-source web application shows that our approach is highly promising: Automatically generated page object methods cover most of the application functionalities and result in readable and meaningful code, which can be very useful to support the creation of more maintainable web test suites.

  • Robula+: an algorithm for generating robust XPath locators for web testing
    Journal of Software: Evolution and Process, 2016
    Co-Authors: Maurizio Leotta, Filippo Ricca, Andrea Stocco, Paolo Tonella
    Abstract:

    Automated test scripts are used with success in many web development projects, so as to automatically verify key functionalities of the web application under test, reveal possible regressions and run a large number of tests in short time. However, the adoption of automated web testing brings advantages but also novel problems, among which the test code fragility problem. During the evolution of the web application, existing test code may easily break and testers have to correct it. In the context of automated DOM-based web testing, one of the major costs for evolving the test code is the manual effort necessary to repair broken web page element locators - lines of source code identifying the web elements e.g. form fields and buttons to interact with.

  • ICWE - Visual vs. DOM-Based Web Locators: An Empirical Study
    Lecture Notes in Computer Science, 2014
    Co-Authors: Maurizio Leotta, Diego Clerissi, Filippo Ricca, Paolo Tonella
    Abstract:

    Automation in Web testing has been successfully supported by DOM-based tools that allow testers to program the interactions of their test cases with the Web application under test. More recently a new generation of visual tools has been proposed where a test case interacts with the Web application by recognising the images of the widgets that can be actioned upon and by asserting the expected visual appearance of the result.

  • ICST - Repairing Selenium test Cases: An Industrial Case Study about Web Page Element Localization
    2013 IEEE Sixth International Conference on Software Testing Verification and Validation, 2013
    Co-Authors: Maurizio Leotta, Diego Clerissi, Filippo Ricca, Cristiano Spadaro
    Abstract:

    This poster presents an industrial case study about test automation and test suite maintenance in the context of Web applications. The Web application under test is a Learning Content Management System (eXact learning LCMS). We analysed the costs associated with the realignment of four equivalent Selenium WebDriver test suites, implemented using the page object pattern and different methods to locate web page elements, to a subsequent release of eXact learning LCMS. In our study, the two ID-based test suites required significantly less maintenance effort than the XPath-based ones.

  • ICST Workshops - Improving test Suites Maintainability with the Page Object Pattern: An Industrial Case Study
    2013 IEEE Sixth International Conference on Software Testing Verification and Validation Workshops, 2013
    Co-Authors: Maurizio Leotta, Diego Clerissi, Filippo Ricca, Cristiano Spadaro
    Abstract:

    The page object pattern is used in the context of web testing for abstracting the application's web pages in order to reduce the coupling between test cases and application under test. This paper reports on an industrial case study in a small Italian company (eXact learning solutions S.p.A.) investigating the potential benefits of adopting the page object pattern to improve the maintainability of Selenium WebDriver test cases. After a maintenance/evolution activity performed on the application under test, we compared two equivalent test suites, one built using the page object pattern and one without it. The results of our case study indicate a strong reduction in terms of time required (by a factor of about three) and number of modified LOCs (by a factor of about eight) to repair the test suite when the page object pattern is used.