The Experts below are selected from a list of 51 Experts worldwide ranked by ideXlab platform

Enda Ridge - One of the best experts on this subject based on the ideXlab platform.

  • Chapter 13 – Testing Data
    Guerrilla Analytics, 2020
    Co-Authors: Enda Ridge
    Abstract:

    This chapter focuses on testing of data. Testing of data involves checking data that the team receives to detect defects. These defects can arise at data extraction, data receipt, and at data load into the team’s Data Manipulation Environment (DME). You will learn about the five ways in which data can be tested in Guerrilla Analytics. These are (1) completeness, (2) correctness, (3) consistency, (4) coherence, and (5) accountability. This chapter will also describe some tips for successful data testing including scopes of tests, storage of test results, common test routines, and automation of testing.

  • Introduction to Testing
    Guerrilla Analytics, 2020
    Co-Authors: Enda Ridge
    Abstract:

    This chapter introduces testing in Guerrilla Analytics projects. It begins with describing where testing fits within the Guerrilla Analytics workflow. We will then discuss the fundamental concept of what it means to test something and why it is important. The areas of Analytics testing will be introduced. You will also learn some tips on testing that can be applied across all these areas.

  • Stage 3: Data Load
    Guerrilla Analytics, 2020
    Co-Authors: Enda Ridge
    Abstract:

    This chapter discusses the Data Load stage of the Guerrilla Analytics workflow. Data Load involves getting data from a receipt location (generally the file system) and loading it into the Data Manipulation Environment (DME). In this chapter, you will learn about the various activities that take place at Data Load. You will learn about the pitfalls and risks in these activities. You will then learn a number of practice tips to mitigate those risks.

  • Stage 4: Analytics Coding to Maintain Data Provenance
    Guerrilla Analytics, 2020
    Co-Authors: Enda Ridge
    Abstract:

    This chapter continues to explore program code for the Analytics stage of the Guerrilla Analytics workflow. In particular, we now focus on how best to manipulate data with program code so that data provenance is maintained. In this chapter you will learn simple tips and tricks that make it much easier to determine where data came from, how they were manipulated and changed, and how they contributed to a final work product.

  • Stage 2: Data Receipt
    Guerrilla Analytics, 2020
    Co-Authors: Enda Ridge
    Abstract:

    Data Receipt is the second stage in the Guerrilla Analytics workflow. It involves delivery of data to the Analytics team and its storage in the team’s data environment. This chapter discusses the pitfalls and risks of data receipt and then recommends tips to mitigate them. At the end of this chapter, you will know how to safely receive data so that data provenance can be maintained.

Edward Curry - One of the best experts on this subject based on the ideXlab platform.

  • emerging principles for Guerrilla Analytics development research in progress
    2012
    Co-Authors: Enda Ridge, Edward Curry
    Abstract:

    Analytics projects come in many forms, from large-scale multi-year projects to projects with small teams lasting just a few weeks. There is a particular type of Analytics project identified by some unique challenges. A team is assembled for the purposes of the project and so team members have not worked together before. The project is short term so there is little opportunity to build capability. Work is often done on client systems requiring the use of limited and perhaps unfamiliar tools. Deadlines are daily or weekly and the requirements can shift repeatedly. Outputs produced in these circumstances will be subject to audit and an expectation of full reproducibility. These are 'Guerrilla Analytics' projects. They necessitate a versatile and fast moving Analytics team that can achieve quick Analytics wins against a large data challenge using lightweight processes and tools. The unique challenges of Guerrilla Analytics necessitate a particular type of data Analytics development process. This paper presents research in progress towards identifying a set of development principles for fast paced Guerrilla Analytics project environments. The paper’s principles cover 4 areas. Data Manipulation principles describe the environment and common services needed by a Guerrilla Analytics team. Data Provenance principles describe how data should be logged, separated and version controlled. Coding and Testing principles describe how code should be structured and outputs tested. All these principles focus on lightweight processes for overcoming the challenges of a Guerrilla Analytics project environment while meeting the Guerrilla Analytics requirement of auditability and reproducibility.