The Experts below are selected from a list of 180282 Experts worldwide ranked by ideXlab platform
Stephen V Stehman - One of the best experts on this subject based on the ideXlab platform.
-
Key issues in rigorous Accuracy Assessment of land cover products
Remote Sensing of Environment, 2019Co-Authors: Stephen V Stehman, Giles M. FoodyAbstract:Abstract Accuracy Assessment and land cover mapping have been inexorably linked throughout the first 50 years of publication of Remote Sensing of Environment. The earliest developers of land-cover maps recognized the importance of evaluating the quality of their maps, and the methods and reporting format of these early Accuracy Assessments included features that would be familiar to practitioners today. Specifically, practitioners have consistently recognized the importance of obtaining high quality reference data to which the map is compared, the need for sampling to collect these reference data, and the role of an error matrix and Accuracy measures derived from the error matrix to summarize the Accuracy information. Over the past half century these techniques have undergone refinements to place Accuracy Assessment on a more scientifically credible footing. We describe the current status of Accuracy Assessment that has emerged from nearly 50 years of practice and identify opportunities for future advances. The article is organized by the three major components of Accuracy Assessment, the sampling design, response design, and analysis, focusing on good practice methodology that contributes to a rigorous, informative, and honest Assessment. The long history of research and applications underlying the current practice of Accuracy Assessment has advanced the field to a mature state. However, documentation of Accuracy Assessment methods needs to be improved to enhance reproducibility and transparency, and improved methods are required to address new challenges created by advanced technology that has expanded the capacity to map land cover extensively in space and intensively in time.
-
Accuracy Assessment of NLCD 2006 land cover and impervious surface
Remote Sensing of Environment, 2013Co-Authors: James D. Wickham, Stephen V Stehman, Leila Gass, Jon Dewitz, Joyce Fry, Timothy G. WadeAbstract:Abstract Release of NLCD 2006 provides the first wall-to-wall land-cover change database for the conterminous United States from Landsat Thematic Mapper (TM) data. Accuracy Assessment of NLCD 2006 focused on four primary products: 2001 land cover, 2006 land cover, land-cover change between 2001 and 2006, and impervious surface change between 2001 and 2006. The Accuracy Assessment was conducted by selecting a stratified random sample of pixels with the reference classification interpreted from multi-temporal high resolution digital imagery. The NLCD Level II (16 classes) overall accuracies for the 2001 and 2006 land cover were 79% and 78%, respectively, with Level II user's accuracies exceeding 80% for water, high density urban, all upland forest classes, shrubland, and cropland for both dates. Level I (8 classes) accuracies were 85% for NLCD 2001 and 84% for NLCD 2006. The high overall and user's accuracies for the individual dates translated into high user's accuracies for the 2001–2006 change reporting themes water gain and loss, forest loss, urban gain, and the no-change reporting themes for water, urban, forest, and agriculture. The main factor limiting higher accuracies for the change reporting themes appeared to be difficulty in distinguishing the context of grass. We discuss the need for more research on land-cover change Accuracy Assessment.
-
Estimating area from an Accuracy Assessment error matrix
Remote Sensing of Environment, 2013Co-Authors: Stephen V StehmanAbstract:Abstract A map of land cover or land-cover change produced from remotely sensed data is linked to estimation of area of land cover or land-cover change via an Accuracy Assessment of the map. A variety of area estimators have been proposed based on different approaches to using the estimated error matrix produced from an Accuracy Assessment along with information available from the map. These estimators include a stratified estimator (where the strata are the map classes), several model-assisted estimators incorporating the map information as auxiliary variables in a variety of different models, and a bias-adjusted estimator that corrects for classification error when area is computed directly from the map. In some cases the same area estimator results from more than one approach. For stratified random sampling with the map classes defining the strata, the model-assisted and bias-adjusted estimators are equivalent to the stratified estimator of area that would typically be used with this sampling design. Thus the commonly used stratified estimator is the lone choice for stratified random sampling. For simple random sampling, the bias-adjusted estimator and a model-assisted difference estimator are equivalent, but other model-assisted options include poststratified (i.e., applying a stratified estimator to data obtained from a simple random sample), ratio, and simple regression estimators. A simulation study demonstrates that for simple random sampling, the poststratified estimator almost always has the smallest variance among these estimators. The only exception to the superior performance of the poststratified estimator occurred when overall Accuracy was very high, the true proportion of area was small (i.e., less than 2%), and the Accuracy Assessment sample size was small ( n = 100). Because the poststratified estimator for simple random sampling is equivalent to the stratified estimator used with stratified random sampling, the stratified estimator provides a unified, simple approach to area estimation for these two commonly used sampling designs.
-
Pixels, blocks of pixels, and polygons: Choosing a spatial unit for thematic Accuracy Assessment
Remote Sensing of Environment, 2011Co-Authors: Stephen V Stehman, James D. WickhamAbstract:Abstract Pixels, blocks of pixels, and polygons are all potentially viable spatial Assessment units for conducting an Accuracy Assessment. We develop a population-based statistical framework to examine how the spatial unit chosen affects the outcome of an Accuracy Assessment. The population is conceptualized as a difference map created by overlaying a complete coverage reference classification and the target map being evaluated. The per-class areas of agreement and disagreement derived from this population are summarized by a population error matrix and Accuracy parameters (e.g., overall, user's and producer's accuracies). The population and values of the Accuracy parameters are strongly affected by the protocols implemented for the response design which include the choice of spatial unit, how within-unit homogeneity is addressed when assigning class labels, and the definition of agreement between the reference and map classification. Several complete coverage populations are used to illustrate how Accuracy results are affected by the spatial unit chosen for the Assessment and also to evaluate how spatial misregistration of the map and reference locations impacts Accuracy results for different spatial units. The sampling design implemented for Accuracy Assessment does not change the population or values of the Accuracy parameters, but the choice of spatial unit will influence decisions regarding use of strata and clusters in the design. A universally best spatial Assessment unit does not exist, so it is critical to recognize how the population, values of the Accuracy parameters, and sampling design are impacted by the choice of spatial unit.
-
Sampling designs for Accuracy Assessment of land cover
International Journal of Remote Sensing, 2009Co-Authors: Stephen V StehmanAbstract:The Accuracy of a land cover classification is the degree to which the map land cover agrees with the reference land cover classification (i.e. ground condition). The basic sampling designs historically implemented for map Accuracy Assessment have served well for the error matrix based analyses traditionally used. But contemporary applications of land cover maps place greater demands on Accuracy Assessment, and sampling designs must be constructed to target objectives such as Accuracy of land cover composition and landscape pattern. Sampling designs differ in their suitability to achieve different objectives, and trade-offs among desirable sampling design criteria must be recognized and accommodated when selecting a design. An overview is presented of the sampling designs used in Accuracy Assessment, and the status of these designs is appraised for meeting current needs. Sampling design features that facilitate multiple-objective Accuracy Assessments are described.
Russell G. Congalton - One of the best experts on this subject based on the ideXlab platform.
-
The Positional Effect in Soft Classification Accuracy Assessment
American Journal of Remote Sensing, 2019Co-Authors: Russell G. CongaltonAbstract:Recent research has included the rapid development of soft classification algorithms and soft classification Accuracy Assessment beyond the traditional hard approaches. However, less consideration has been given to whether conditions and assumptions generated for the hard classification Accuracy Assessment are appropriate for the soft one. Positional error is one of the most significant uncertainties that need to be considered. This research examined the impacts of positional errors on the Accuracy measures derived from the soft error matrix using NLCD 2011 as reference data and several coarser maps generated from NLCD 2011 as classification maps at the spatial resolutions of 150m, 300m, 600m, and 900m. Eight study sites, with a spatial extent of 180km×180km, of different landscape characteristics were investigated using a two-level classification scheme. Results showed that with existing registration accuracies achieved by current global land cover mapping, the errors in overall Accuracy (OA-error) were 2.13% -39.98% and 2.53%-48.82% for the 8 and 15 classes, respectively and the errors in Kappa (Kappa-error) were 6.64%-57.09% and 7.08%-58.81% for the 8 and 15 classes, respectively if soft classifications were implemented based on images where spatial resolutions varied from 150m to 900m. More complex landscape characteristics and classes in the classification scheme produced a greater impact of the positional error on the Accuracy measures. To keep both OA-error and Kappa-error under 10 percent, the average required registration Accuracy should achieve 0.1 pixels. This paper strongly recommends the addition of uncertainty analysis due to positional error in future global land cover mapping.
-
Issues with Large Area Thematic Accuracy Assessment for Mapping Cropland Extent: A Tale of Three Continents
Remote Sensing, 2017Co-Authors: Kamini Yadav, Russell G. CongaltonAbstract:Accurate, consistent and timely cropland information over large areas is critical to solve food security issues. To predict and respond to food insecurity, global cropland products are readily available from coarse and medium spatial resolution earth observation data. However, while the use of satellite imagery has great potential to identify cropland areas and their specific types, the full potential of this imagery has yet to be realized due to variability of croplands in different regions. Despite recent calls for statistically robust and transparent Accuracy Assessment, more attention regarding the Accuracy Assessment of large area cropland maps is still needed. To conduct a valid Assessment of cropland maps, different strategies, issues and constraints need to be addressed depending upon various conditions present in each continent. This study specifically focused on dealing with some specific issues encountered when assessing the cropland extent of North America (confined to the United States), Africa and Australia. The process of Accuracy Assessment was performed using a simple random sampling design employed within defined strata (i.e., Agro-Ecological Zones (AEZ’s) for the US and Africa and a buffer zone approach around the cropland areas of Australia. Continent-specific sample analysis was performed to ensure that an appropriate reference data set was used to generate a valid error matrix indicative of the actual cropland proportion. Each Accuracy Assessment was performed within the homogenous regions (i.e., strata) of different continents using different sources of reference data to produce rigorous and valid Accuracy results. The results indicate that continent-specific modified Assessments performed for the three selected continents demonstrate that the Accuracy Assessment can be easily accomplished for a large area such as the US that has extensive availability of reference data while more modifications were needed in the sampling design for other continents that had little to no reference data and other constraints. Each continent provided its own unique challenges and opportunities. Therefore, this paper describes a tale of these three continents providing recommendations to adapt Accuracy Assessment strategies and methodologies for validating global cropland extent maps.
-
the impact of positional errors on soft classification Accuracy Assessment a simulation analysis
Remote Sensing, 2015Co-Authors: Russell G. Congalton, Yaozhong PanAbstract:Validating or accessing the Accuracy of soft classification maps has rapidly developed over the past few years. This Assessment employs a soft error matrix as generalized from the traditional, hard classification error matrix. However, the impact of positional error on the soft classification is uncertain and whether the well-accepted half-pixel registration Accuracy is suitable for the soft classification Accuracy Assessment is unknown. In this paper, a simulation analysis was conducted to examine the influence of positional error on the overall Accuracy (OA) and kappa in soft classification Accuracy Assessment under different landscape conditions (i.e., spatial characteristics and spatial resolutions). Results showed that with positional error ranging from 0 to 3 soft pixels, the OA-error varied from 0 to 44.6 percent while the kappa-error varied from 0 to 93.7 percent. Landscape conditions with smaller mean patch size (MPS) and greater fragmentation produced greater positional error impact on the Accuracy measures at spatial resolutions of 1 and 2 unit distances. However, this trend did not hold for spatial resolutions of 5 and 10 unit distances. A half of a pixel was not sufficient to keep the overall Accuracy error and kappa error under 10 percent. The results indicate that for soft classification Accuracy Assessment the requirement for registration Accuracy is higher and depends greatly on the landscape characteristics. There is a great need to consider positional error for validating soft classification maps of different spatial resolutions.
-
Thematic and positional Accuracy Assessment of digital remotely sensed data
2007Co-Authors: Russell G. CongaltonAbstract:Accuracy Assessment or validation has become a standard component of any land cover or vegetation map derived from remotely sensed data. Knowing the Accuracy of the map is vital to any decisionmaking performed using that map. The process of assessing the map Accuracy is time consuming and expensive. It is very important that the procedure be well thought out and carefully planned to be as efficient as possible. This paper presents a brief review of the current methods used in thematic map Accuracy Assessment. A discussion of positional error is included as it is impossible to assess thematic Accuracy without carefully considering positional Accuracy.
Bakhtiar Feizizadeh - One of the best experts on this subject based on the ideXlab platform.
-
a novel approach of fuzzy dempster shafer theory for spatial uncertainty analysis and Accuracy Assessment of object based image classification
IEEE Geoscience and Remote Sensing Letters, 2018Co-Authors: Bakhtiar FeizizadehAbstract:Accuracy Assessment is a fundamental step in remote-sensing image processing. The Accuracy Assessment techniques aim to compute classification Accuracy and characterize errors, and can, thus, be used to refine the classification or estimates derived from the Assessment itself. With regard to their technical capabilities, these techniques have been criticized for their inherent uncertainty and inability to evaluate image classification accuracies. To overcome this issue, the main objective of this letter was to introduce a new approach for the Accuracy Assessment of object-based image analysis (OBIA). To this end, an integrated approach of fuzzy synthetic evaluation and Dempster–Shafer theory (FSE-DST) was adapted and proposed as an effective approach for object-based image classification Accuracy Assessment. Two experiments were established to examine the capability of the proposed approach. OBIA was applied to develop a land-use land-cover map of Ahar city and the Ousko area. The proposed FSE-DST was applied for a spatially explicit Accuracy Assessment. Results indicate that FSE-DST can be effectively applied in spatial Accuracy Assessments for OBIA and for spatial Accuracy Assessments in remote-sensing-based classifications. The results of this letter are important to the development of OBIA and can serve as the basis for progressive research in remote sensing by supporting future researchers in obtaining more accurate results from OBIA-based classifications and spatially analyzing the reliability of results.
-
A Novel Approach of Fuzzy Dempster–Shafer Theory for Spatial Uncertainty Analysis and Accuracy Assessment of Object-Based Image Classification
IEEE Geoscience and Remote Sensing Letters, 2018Co-Authors: Bakhtiar FeizizadehAbstract:Accuracy Assessment is a fundamental step in remote-sensing image processing. The Accuracy Assessment techniques aim to compute classification Accuracy and characterize errors, and can, thus, be used to refine the classification or estimates derived from the Assessment itself. With regard to their technical capabilities, these techniques have been criticized for their inherent uncertainty and inability to evaluate image classification accuracies. To overcome this issue, the main objective of this letter was to introduce a new approach for the Accuracy Assessment of object-based image analysis (OBIA). To this end, an integrated approach of fuzzy synthetic evaluation and Dempster–Shafer theory (FSE-DST) was adapted and proposed as an effective approach for object-based image classification Accuracy Assessment. Two experiments were established to examine the capability of the proposed approach. OBIA was applied to develop a land-use land-cover map of Ahar city and the Ousko area. The proposed FSE-DST was applied for a spatially explicit Accuracy Assessment. Results indicate that FSE-DST can be effectively applied in spatial Accuracy Assessments for OBIA and for spatial Accuracy Assessments in remote-sensing-based classifications. The results of this letter are important to the development of OBIA and can serve as the basis for progressive research in remote sensing by supporting future researchers in obtaining more accurate results from OBIA-based classifications and spatially analyzing the reliability of results.
Raymond L Czaplewski - One of the best experts on this subject based on the ideXlab platform.
-
Accuracy Assessment with complex sampling designs
2010Co-Authors: Raymond L CzaplewskiAbstract:A reliable Accuracy Assessment of remotely sensed geospatial data requires a sufficiently large probability sample of expensive reference data. Complex sampling designs reduce cost or increase precision, especially with regional, continental and global projects. The General Restriction (GR) Estimator and the Recursive Restriction (RR) Estimator separate a complex sample survey into simple statistical components, each of which is sequentially combined into the final estimate. GR and RR produce a design-consistent Empirical Best Linear Unbiased Estimator (EBLUE) for any sample survey design, regardless of its complexity.
-
Accuracy Assessment of Maps of Forest Condition
Remote Sensing of Forest Environments, 2003Co-Authors: Raymond L CzaplewskiAbstract:No thematic map is perfect. Some pixels or polygons are not accurately classified, no matter how well the map is crafted. Therefore, thematic maps need metadata that sufficiently characterize the nature and degree of these imperfections. To decision-makers, an Accuracy Assessment helps judge the risks of using imperfect geospatial data. To analysts, an Accuracy Assessment helps describe the reliability of the map for geospatial analyses and modeling, and the distribution of different types of “true” land cover within each mapped category. To producers of thematic maps, an Accuracy Assessment measures the degree of technical success for alternative algorithms or techniques. To project managers, an Accuracy Assessment helps determine contract compliance or measure performance of technical staff.
-
COMBINING Accuracy Assessment OF LAND-COVER MAPS WITH ENVIRONMENTAL MONITORING PROGRAMS
Environmental Monitoring and Assessment, 2000Co-Authors: Stephen V Stehman, Raymond L Czaplewski, Sarah M. Nusser, Limin Yang, Zhiliang ZhuAbstract:A scientifically valid Accuracy Assessment of a large-area, land-cover map is expensive. Environmental monitoring programs offer a potential source of data to partially defray the cost of Accuracy Assessment while still maintaining the statistical validity. In this article, three general strategies for combining Accuracy Assessment and environmental monitoring protocols are described. These strategies range from a fully integrated Accuracy Assessment and environmental monitoring protocol, to one in which the protocols operate nearly independently. For all three strategies, features critical to using monitoring data for Accuracy Assessment include compatibility of the land-cover classification schemes, precisely co-registered sample data, and spatial and temporal compatibility of the map and reference data. Two monitoring programs, the National Resources Inventory (NRI) and the Forest Inventory and Monitoring (FIM), are used to illustrate important features for implementing a combined protocol.
-
design and analysis for thematic map Accuracy Assessment fundamental principles
Remote Sensing of Environment, 1998Co-Authors: Stephen V Stehman, Raymond L CzaplewskiAbstract:Abstract Before being used in scientific investigations and policy decisions, thematic maps constructed from remotely sensed data should be subjected to a statistically rigorous Accuracy Assessment. The three basic components of an Accuracy Assessment are: 1) the sampling design used to select the reference sample; 2) the response design used to obtain the reference land-cover classification for each sampling unit; and 3) the estimation and analysis procedures. We discuss options available for each of these components. A statistically rigorous Assessment requires both a probability sampling design and statistically consistent estimators of Accuracy parameters, along with a response design determined in accordance with features of the mapping and classification process such as the land-cover classification scheme, minimum mapping unit, and spatial scale of the mapping.
Giles M. Foody - One of the best experts on this subject based on the ideXlab platform.
-
Key issues in rigorous Accuracy Assessment of land cover products
Remote Sensing of Environment, 2019Co-Authors: Stephen V Stehman, Giles M. FoodyAbstract:Abstract Accuracy Assessment and land cover mapping have been inexorably linked throughout the first 50 years of publication of Remote Sensing of Environment. The earliest developers of land-cover maps recognized the importance of evaluating the quality of their maps, and the methods and reporting format of these early Accuracy Assessments included features that would be familiar to practitioners today. Specifically, practitioners have consistently recognized the importance of obtaining high quality reference data to which the map is compared, the need for sampling to collect these reference data, and the role of an error matrix and Accuracy measures derived from the error matrix to summarize the Accuracy information. Over the past half century these techniques have undergone refinements to place Accuracy Assessment on a more scientifically credible footing. We describe the current status of Accuracy Assessment that has emerged from nearly 50 years of practice and identify opportunities for future advances. The article is organized by the three major components of Accuracy Assessment, the sampling design, response design, and analysis, focusing on good practice methodology that contributes to a rigorous, informative, and honest Assessment. The long history of research and applications underlying the current practice of Accuracy Assessment has advanced the field to a mature state. However, documentation of Accuracy Assessment methods needs to be improved to enhance reproducibility and transparency, and improved methods are required to address new challenges created by advanced technology that has expanded the capacity to map land cover extensively in space and intensively in time.
-
sample size determination for image classification Accuracy Assessment and comparison
International Journal of Remote Sensing, 2009Co-Authors: Giles M. FoodyAbstract:Many factors influence the quality and value of a classification Accuracy Assessment and evaluation programme. This paper focuses on the size of the testing set(s) used with particular regard to the impacts on Accuracy Assessment and comparison. Testing set size is important as the use of an inappropriately large or small sample could lead to limited and sometimes erroneous Assessments of Accuracy and of differences in Accuracy. Here, some of the basic statistical principles of sample size determination are outlined, including a discussion of Type II errors and their control. The paper provides a discussion on some of the basic issues of sample size determination for Accuracy Assessment and includes factors linked to Accuracy comparison. With the latter, the researcher should specify the effect size (minimum meaningful difference in Accuracy), significance level and power used in an analysis and ideally also fit confidence limits to derived estimates. This will help design a study and aid the use of appro...
-
harshness in image classification Accuracy Assessment
Journal of remote sensing, 2008Co-Authors: Giles M. FoodyAbstract:Thematic mapping via a classification analysis is one of the most common applications of remote sensing. The Accuracy of image classifications is, however, often viewed negatively. Here, it is suggested that the approach to the evaluation of image classification Accuracy typically adopted in remote sensing may often be unfair, commonly being rather harsh and misleading. It is stressed that the widely used target Accuracy of 85% can be inappropriate and that the approach to Accuracy Assessment adopted commonly in remote sensing is pessimistically biased. Moreover, the maps produced by other communities, which are often used unquestioningly, may have a low Accuracy if evaluated from the standard perspective adopted in remote sensing. A greater awareness of the problems encountered in Accuracy Assessment may help ensure that perceptions of classification Accuracy are realistic and reduce unfair criticism of thematic maps derived from remote sensing.
-
Status of land cover classification Accuracy Assessment
Remote Sensing of Environment, 2002Co-Authors: Giles M. FoodyAbstract:The production of thematic maps, such as those depicting land cover, using an image classification is one of the most common applications of remote sensing. Considerable research has been directed at the various components of the mapping process, including the Assessment of Accuracy. This paper briefly reviews the background and methods of classification Accuracy Assessment that are commonly used and recommended in the research literature. It is, however, evident that the research community does not universally adopt the approaches that are often recommended to it, perhaps a reflection of the problems associated with Accuracy Assessment, and typically fails to achieve the Accuracy targets commonly specified. The community often tends to use, unquestioningly, techniques based on the confusion matrix for which the correct application and interpretation requires the satisfaction of often untenable assumptions (e.g., perfect coregistration of data sets) and the provision of rarely conveyed information (e.g., sampling design for ground data acquisition). Eight broad problem areas that currently limit the ability to appropriately assess, document, and use the Accuracy of thematic maps derived from remote sensing are explored. The implications of these problems are that it is unlikely that a single standardized method of Accuracy Assessment and reporting can be identified, but some possible directions for future research that may facilitate Accuracy Assessment are highlighted.
