Accuracy Assessment - Explore the Science & Experts | ideXlab

Scan Science and Technology

Contact Leading Edge Experts & Companies

Accuracy Assessment

The Experts below are selected from a list of 180282 Experts worldwide ranked by ideXlab platform

Accuracy Assessment – Free Register to Access Experts & Abstracts

Stephen V Stehman – One of the best experts on this subject based on the ideXlab platform.

  • Key issues in rigorous Accuracy Assessment of land cover products
    Remote Sensing of Environment, 2019
    Co-Authors: Stephen V Stehman, Giles M. Foody
    Abstract:

    Abstract Accuracy Assessment and land cover mapping have been inexorably linked throughout the first 50 years of publication of Remote Sensing of Environment. The earliest developers of land-cover maps recognized the importance of evaluating the quality of their maps, and the methods and reporting format of these early Accuracy Assessments included features that would be familiar to practitioners today. Specifically, practitioners have consistently recognized the importance of obtaining high quality reference data to which the map is compared, the need for sampling to collect these reference data, and the role of an error matrix and Accuracy measures derived from the error matrix to summarize the Accuracy information. Over the past half century these techniques have undergone refinements to place Accuracy Assessment on a more scientifically credible footing. We describe the current status of Accuracy Assessment that has emerged from nearly 50 years of practice and identify opportunities for future advances. The article is organized by the three major components of Accuracy Assessment, the sampling design, response design, and analysis, focusing on good practice methodology that contributes to a rigorous, informative, and honest Assessment. The long history of research and applications underlying the current practice of Accuracy Assessment has advanced the field to a mature state. However, documentation of Accuracy Assessment methods needs to be improved to enhance reproducibility and transparency, and improved methods are required to address new challenges created by advanced technology that has expanded the capacity to map land cover extensively in space and intensively in time.

  • Accuracy Assessment of NLCD 2006 land cover and impervious surface
    Remote Sensing of Environment, 2013
    Co-Authors: James D. Wickham, Stephen V Stehman, Leila Gass, Jon Dewitz, Joyce Fry, Timothy G. Wade
    Abstract:

    Abstract Release of NLCD 2006 provides the first wall-to-wall land-cover change database for the conterminous United States from Landsat Thematic Mapper (TM) data. Accuracy Assessment of NLCD 2006 focused on four primary products: 2001 land cover, 2006 land cover, land-cover change between 2001 and 2006, and impervious surface change between 2001 and 2006. The Accuracy Assessment was conducted by selecting a stratified random sample of pixels with the reference classification interpreted from multi-temporal high resolution digital imagery. The NLCD Level II (16 classes) overall accuracies for the 2001 and 2006 land cover were 79% and 78%, respectively, with Level II user’s accuracies exceeding 80% for water, high density urban, all upland forest classes, shrubland, and cropland for both dates. Level I (8 classes) accuracies were 85% for NLCD 2001 and 84% for NLCD 2006. The high overall and user’s accuracies for the individual dates translated into high user’s accuracies for the 2001–2006 change reporting themes water gain and loss, forest loss, urban gain, and the no-change reporting themes for water, urban, forest, and agriculture. The main factor limiting higher accuracies for the change reporting themes appeared to be difficulty in distinguishing the context of grass. We discuss the need for more research on land-cover change Accuracy Assessment.

  • Estimating area from an Accuracy Assessment error matrix
    Remote Sensing of Environment, 2013
    Co-Authors: Stephen V Stehman
    Abstract:

    Abstract A map of land cover or land-cover change produced from remotely sensed data is linked to estimation of area of land cover or land-cover change via an Accuracy Assessment of the map. A variety of area estimators have been proposed based on different approaches to using the estimated error matrix produced from an Accuracy Assessment along with information available from the map. These estimators include a stratified estimator (where the strata are the map classes), several model-assisted estimators incorporating the map information as auxiliary variables in a variety of different models, and a bias-adjusted estimator that corrects for classification error when area is computed directly from the map. In some cases the same area estimator results from more than one approach. For stratified random sampling with the map classes defining the strata, the model-assisted and bias-adjusted estimators are equivalent to the stratified estimator of area that would typically be used with this sampling design. Thus the commonly used stratified estimator is the lone choice for stratified random sampling. For simple random sampling, the bias-adjusted estimator and a model-assisted difference estimator are equivalent, but other model-assisted options include poststratified (i.e., applying a stratified estimator to data obtained from a simple random sample), ratio, and simple regression estimators. A simulation study demonstrates that for simple random sampling, the poststratified estimator almost always has the smallest variance among these estimators. The only exception to the superior performance of the poststratified estimator occurred when overall Accuracy was very high, the true proportion of area was small (i.e., less than 2%), and the Accuracy Assessment sample size was small ( n  = 100). Because the poststratified estimator for simple random sampling is equivalent to the stratified estimator used with stratified random sampling, the stratified estimator provides a unified, simple approach to area estimation for these two commonly used sampling designs.

Russell G. Congalton – One of the best experts on this subject based on the ideXlab platform.

  • The Positional Effect in Soft Classification Accuracy Assessment
    American Journal of Remote Sensing, 2019
    Co-Authors: Russell G. Congalton
    Abstract:

    Recent research has included the rapid development of soft classification algorithms and soft classification Accuracy Assessment beyond the traditional hard approaches. However, less consideration has been given to whether conditions and assumptions generated for the hard classification Accuracy Assessment are appropriate for the soft one. Positional error is one of the most significant uncertainties that need to be considered. This research examined the impacts of positional errors on the Accuracy measures derived from the soft error matrix using NLCD 2011 as reference data and several coarser maps generated from NLCD 2011 as classification maps at the spatial resolutions of 150m, 300m, 600m, and 900m. Eight study sites, with a spatial extent of 180km×180km, of different landscape characteristics were investigated using a two-level classification scheme. Results showed that with existing registration accuracies achieved by current global land cover mapping, the errors in overall Accuracy (OA-error) were 2.13% -39.98% and 2.53%-48.82% for the 8 and 15 classes, respectively and the errors in Kappa (Kappa-error) were 6.64%-57.09% and 7.08%-58.81% for the 8 and 15 classes, respectively if soft classifications were implemented based on images where spatial resolutions varied from 150m to 900m. More complex landscape characteristics and classes in the classification scheme produced a greater impact of the positional error on the Accuracy measures. To keep both OA-error and Kappa-error under 10 percent, the average required registration Accuracy should achieve 0.1 pixels. This paper strongly recommends the addition of uncertainty analysis due to positional error in future global land cover mapping.

  • Issues with Large Area Thematic Accuracy Assessment for Mapping Cropland Extent: A Tale of Three Continents
    Remote Sensing, 2017
    Co-Authors: Kamini Yadav, Russell G. Congalton
    Abstract:

    Accurate, consistent and timely cropland information over large areas is critical to solve food security issues. To predict and respond to food insecurity, global cropland products are readily available from coarse and medium spatspatial resolution earth observation data. However, while the use of satellite imagery has great potential to identify cropland areas and their specific types, the full potential of this imagery has yet to be realized due to variability of croplands in different regions. Despite recent calls for statistically robust and transparent Accuracy Assessment, more attention regarding the Accuracy Assessment of large area cropland maps is still needed. To conduct a valid Assessment of cropland maps, different strategies, issues and constraints need to be addressed depending upon various conditions present in each continent. This study specifically focused on dealing with some specific issues encountered when assessing the cropland extent of North America (confined to the United States), Africa and Australia. The process of Accuracy Assessment was performed using a simple random sampling design employed within defined strata (i.e., Agro-Ecological Zones (AEZ’s) for the US and Africa and a buffer zone approach around the cropland areas of Australia. Continent-specific sample analysis was performed to ensure that an appropriate reference data set was used to generate a valid error matrix indicative of the actual cropland proportion. Each Accuracy Assessment was performed within the homogenous regions (i.e., strata) of different continents using different sources of reference data to produce rigorous and valid Accuracy results. The results indicate that continent-specific modified Assessments performed for the three selected continents demonstrate that the Accuracy Assessment can be easily accomplished for a large area such as the US that has extensive availability of reference data while more modifications were needed in the sampling design for other continents that had little to no reference data and other constraints. Each continent provided its own unique challenges and opportunities. Therefore, this paper describes a tale of these three continents providing recommendations to adapt Accuracy Assessment strategies and methodologies for validating global cropland extent maps.

  • the impact of positional errors on soft classification Accuracy Assessment a simulation analysis
    Remote Sensing, 2015
    Co-Authors: Russell G. Congalton, Yaozhong Pan
    Abstract:

    Validating or accessing the Accuracy of soft classification maps has rapidly developed over the past few years. This Assessment employs a soft error matrix as generalized from the traditional, hard classification error matrix. However, the impact of positional error on the soft classification is uncertain and whether the well-accepted half-pixel registration Accuracy is suitable for the soft classification Accuracy Assessment is unknown. In this paper, a simulation analysis was conducted to examine the influence of positional error on the overall Accuracy (OA) and kappa in soft classification Accuracy Assessment under different landscape conditions (i.e., spatial characteristics and spatial resolutions). Results showed that with positional error ranging from 0 to 3 soft pixels, the OA-error varied from 0 to 44.6 percent while the kappa-error varied from 0 to 93.7 percent. Landscape conditions with smaller mean patch size (MPS) and greater fragmentation produced greater positional error impact on the Accuracy measures at spatial resolutions of 1 and 2 unit distances. However, this trend did not hold for spatial resolutions of 5 and 10 unit distances. A half of a pixel was not sufficient to keep the overall Accuracy error and kappa error under 10 percent. The results indicate that for soft classification Accuracy Assessment the requirement for registration Accuracy is higher and depends greatly on the landscape characteristics. There is a great need to consider positional error for validating soft classification maps of different spatial resolutions.

Bakhtiar Feizizadeh – One of the best experts on this subject based on the ideXlab platform.

  • a novel approach of fuzzy dempster shafer theory for spatial uncertainty analysis and Accuracy Assessment of object based image classification
    IEEE Geoscience and Remote Sensing Letters, 2018
    Co-Authors: Bakhtiar Feizizadeh
    Abstract:

    Accuracy Assessment is a fundamental step in remote-sensing image processing. The Accuracy Assessment techniques aim to compute classification Accuracy and characterize errors, and can, thus, be used to refine the classification or estimates derived from the Assessment itself. With regard to their technical capabilities, these techniques have been criticized for their inherent uncertainty and inability to evaluate image classification accuracies. To overcome this issue, the main objective of this letter was to introduce a new approach for the Accuracy Assessment of object-based image analysis (OBIA). To this end, an integrated approach of fuzzy synthetic evaluation and Dempster–Shafer theory (FSE-DST) was adapted and proposed as an effective approach for object-based image classification Accuracy Assessment. Two experiments were established to examine the capability of the proposed approach. OBIA was applied to develop a land-use land-cover map of Ahar city and the Ousko area. The proposed FSE-DST was applied for a spatially explicit Accuracy Assessment. Results indicate that FSE-DST can be effectively applied in spatial Accuracy Assessments for OBIA and for spatial Accuracy Assessments in remote-sensing-based classifications. The results of this letter are important to the development of OBIA and can serve as the basis for progressive research in remote sensing by supporting future researchers in obtaining more accurate results from OBIA-based classifications and spatially analyzing the reliability of results.

  • A Novel Approach of Fuzzy Dempster–Shafer Theory for Spatial Uncertainty Analysis and Accuracy Assessment of Object-Based Image Classification
    IEEE Geoscience and Remote Sensing Letters, 2018
    Co-Authors: Bakhtiar Feizizadeh
    Abstract:

    Accuracy Assessment is a fundamental step in remote-sensing image processing. The Accuracy Assessment techniques aim to compute classification Accuracy and characterize errors, and can, thus, be used to refine the classification or estimates derived from the Assessment itself. With regard to their technical capabilities, these techniques have been criticized for their inherent uncertainty and inability to evaluate image classification accuracies. To overcome this issue, the main objective of this letter was to introduce a new approach for the Accuracy Assessment of object-based image analysis (OBIA). To this end, an integrated approach of fuzzy synthetic evaluation and Dempster–Shafer theory (FSE-DST) was adapted and proposed as an effective approach for object-based image classification Accuracy Assessment. Two experiments were established to examine the capability of the proposed approach. OBIA was applied to develop a land-use land-cover map of Ahar city and the Ousko area. The proposed FSE-DST was applied for a spatially explicit Accuracy Assessment. Results indicate that FSE-DST can be effectively applied in spatial Accuracy Assessments for OBIA and for spatial Accuracy Assessments in remote-sensing-based classifications. The results of this letter are important to the development of OBIA and can serve as the basis for progressive research in remote sensing by supporting future researchers in obtaining more accurate results from OBIA-based classifications and spatially analyzing the reliability of results.

Raymond L Czaplewski – One of the best experts on this subject based on the ideXlab platform.

  • Accuracy Assessment with complex sampling designs
    , 2010
    Co-Authors: Raymond L Czaplewski
    Abstract:

    A reliable Accuracy Assessment of remotely sensed geospatial data requires a sufficiently large probability sample of expensive reference data. Complex sampling designs reduce cost or increase precision, especially with regional, continental and global projects. The General Restriction (GR) Estimator and the Recursive Restriction (RR) Estimator separate a complex sample survey into simple statistical components, each of which is sequentially combined into the final estimate. GR and RR produce a design-consistent Empirical Best Linear Unbiased Estimator (EBLUE) for any sample survey design, regardless of its complexity.

  • Accuracy Assessment of Maps of Forest Condition
    Remote Sensing of Forest Environments, 2003
    Co-Authors: Raymond L Czaplewski
    Abstract:

    No thematic map is perfect. Some pixels or polygons are not accurately classified, no matter how well the map is crafted. Therefore, thematic maps need metadata that sufficiently characterize the nature and degree of these imperfections. To decision-makers, an Accuracy Assessment helps judge the risks of using imperfect geospatial data. To analysts, an Accuracy Assessment helps describe the reliability of the map for geospatial analyses and modeling, and the distribution of different types of “true” land cover within each mapped category. To producers of thematic maps, an Accuracy Assessment measures the degree of technical success for alternative algorithms or techniques. To project managers, an Accuracy Assessment helps determine contract compliance or measure performance of technical staff.

  • COMBINING Accuracy Assessment OF LAND-COVER MAPS WITH ENVIRONMENTAL MONITORING PROGRAMS
    Environmental Monitoring and Assessment, 2000
    Co-Authors: Stephen V Stehman, Raymond L Czaplewski, Sarah M. Nusser, Limin Yang, Zhiliang Zhu
    Abstract:

    A scientifically valid Accuracy Assessment of a large-area, land-cover map is expensive. Environmental monitoring programs offer a potential source of data to partially defray the cost of Accuracy Assessment while still maintaining the statistical validity. In this article, three general strategies for combining Accuracy Assessment and environmental monitoring protocols are described. These strategies range from a fully integrated Accuracy Assessment and environmental monitoring protocol, to one in which the protocols operate nearly independently. For all three strategies, features critical to using monitoring data for Accuracy Assessment include compatibility of the land-cover classification schemes, precisely co-registered sample data, and spatial and temporal compatibility of the map and reference data. Two monitoring programs, the National Resources Inventory (NRI) and the Forest Inventory and Monitoring (FIM), are used to illustrate important features for implementing a combined protocol.

Giles M. Foody – One of the best experts on this subject based on the ideXlab platform.

  • Key issues in rigorous Accuracy Assessment of land cover products
    Remote Sensing of Environment, 2019
    Co-Authors: Stephen V Stehman, Giles M. Foody
    Abstract:

    Abstract Accuracy Assessment and land cover mapping have been inexorably linked throughout the first 50 years of publication of Remote Sensing of Environment. The earliest developers of land-cover maps recognized the importance of evaluating the quality of their maps, and the methods and reporting format of these early Accuracy Assessments included features that would be familiar to practitioners today. Specifically, practitioners have consistently recognized the importance of obtaining high quality reference data to which the map is compared, the need for sampling to collect these reference data, and the role of an error matrix and Accuracy measures derived from the error matrix to summarize the Accuracy information. Over the past half century these techniques have undergone refinements to place Accuracy Assessment on a more scientifically credible footing. We describe the current status of Accuracy Assessment that has emerged from nearly 50 years of practice and identify opportunities for future advances. The article is organized by the three major components of Accuracy Assessment, the sampling design, response design, and analysis, focusing on good practice methodology that contributes to a rigorous, informative, and honest Assessment. The long history of research and applications underlying the current practice of Accuracy Assessment has advanced the field to a mature state. However, documentation of Accuracy Assessment methods needs to be improved to enhance reproducibility and transparency, and improved methods are required to address new challenges created by advanced technology that has expanded the capacity to map land cover extensively in space and intensively in time.

  • sample size determination for image classification Accuracy Assessment and comparison
    International Journal of Remote Sensing, 2009
    Co-Authors: Giles M. Foody
    Abstract:

    Many factors influence the quality and value of a classification Accuracy Assessment and evaluation programme. This paper focuses on the size of the testing set(s) used with particular regard to the impacts on Accuracy Assessment and comparison. Testing set size is important as the use of an inappropriately large or small sample could lead to limited and sometimes erroneous Assessments of Accuracy and of differences in Accuracy. Here, some of the basic statistical principles of sample sizesize determination are outlined, including a discussion of Type II errors and their control. The paper provides a discussion on some of the basic issues of sample sizesize determination for Accuracy Assessment and includes factors linked to Accuracy comparison. With the latter, the researcher should specify the effect size (minimum meaningful difference in Accuracy), significance level and power used in an analysis and ideally also fit confidence limits to derived estimates. This will help design a study and aid the use of appro…

  • harshness in image classification Accuracy Assessment
    Journal of remote sensing, 2008
    Co-Authors: Giles M. Foody
    Abstract:

    Thematic mapping via a classification analysis is one of the most common applications of remote sensing. The Accuracy of image classifications is, however, often viewed negatively. Here, it is suggested that the approach to the evaluation of image classification Accuracy typically adopted in remote sensing may often be unfair, commonly being rather harsh and misleading. It is stressed that the widely used target Accuracy of 85% can be inappropriate and that the approach to Accuracy Assessment adopted commonly in remote sensing is pessimistically biased. Moreover, the maps produced by other communities, which are often used unquestioningly, may have a low Accuracy if evaluated from the standard perspective adopted in remote sensing. A greater awareness of the problems encountered in Accuracy Assessment may help ensure that perceptions of classification Accuracy are realistic and reduce unfair criticism of thematic maps derived from remote sensing.