Data Quality Issue

14,000,000 Leading Edge Experts on the ideXlab platform

Scan Science and Technology

Contact Leading Edge Experts & Companies

Scan Science and Technology

Contact Leading Edge Experts & Companies

The Experts below are selected from a list of 60 Experts worldwide ranked by ideXlab platform

Yilu Liu - One of the best experts on this subject based on the ideXlab platform.

  • Data Quality Issues for synchrophasor applications Part I: a review
    Journal of Modern Power Systems and Clean Energy, 2016
    Co-Authors: Can Huang, Zhuohong Pan, Jiahui Guo, Dao Zhou, Fangxing Li, Yong Liu, Yilu Liu
    Abstract:

    Synchrophasor systems, providing low-latency, high-precision, and time-synchronized measurements to enhance power grid performances, are deployed globally. However, the synchrophasor system as a physical network, involves communication constraints and Data Quality Issues, which will impact or even disable certain synchrophasor applications. This work investigates the Data Quality Issue for synchrophasor applications. In Part I, the standards of synchrophasor systems and the classifications and Data Quality requirements of synchrophasor applications are reviewed. Also, the actual events of synchronization signal accuracy, synchrophasor Data loss, and latency are counted and analyzed. The review and statistics are expected to provide an overall picture of Data accuracy, loss, and latency Issues for synchrophasor applications.

  • Data Quality Issues for synchrophasor applications Part II: problem formulation and potential solutions
    Journal of Modern Power Systems and Clean Energy, 2016
    Co-Authors: Can Huang, Lingwei Zhan, Qinran Hu, Dao Zhou, Fangxing Li, Yao Xu, Yilu Liu
    Abstract:

    This work investigates the Data Quality Issue for synchrophasor applications, and pays particular attention to synchronization signal loss and synchrophasor Data loss events. First, the historical synchronization signal loss events are analyzed and the potential reasons and solutions are discussed. Then, the scenario of a small amount of synchrophasor Data loss is studied and a Lagrange interpolating polynomial method is used to adaptively estimate the incomplete and missing Data. The performance of proposed method is demonstrated with simulation results. Specifically, the proposed method considers the trade-off between the estimation accuracy and the hardware cost, and could be efficiently employed in reality.

Can Huang - One of the best experts on this subject based on the ideXlab platform.

  • Data Quality Issues for synchrophasor applications Part I: a review
    Journal of Modern Power Systems and Clean Energy, 2016
    Co-Authors: Can Huang, Zhuohong Pan, Jiahui Guo, Dao Zhou, Fangxing Li, Yong Liu, Yilu Liu
    Abstract:

    Synchrophasor systems, providing low-latency, high-precision, and time-synchronized measurements to enhance power grid performances, are deployed globally. However, the synchrophasor system as a physical network, involves communication constraints and Data Quality Issues, which will impact or even disable certain synchrophasor applications. This work investigates the Data Quality Issue for synchrophasor applications. In Part I, the standards of synchrophasor systems and the classifications and Data Quality requirements of synchrophasor applications are reviewed. Also, the actual events of synchronization signal accuracy, synchrophasor Data loss, and latency are counted and analyzed. The review and statistics are expected to provide an overall picture of Data accuracy, loss, and latency Issues for synchrophasor applications.

  • Data Quality Issues for synchrophasor applications Part II: problem formulation and potential solutions
    Journal of Modern Power Systems and Clean Energy, 2016
    Co-Authors: Can Huang, Lingwei Zhan, Qinran Hu, Dao Zhou, Fangxing Li, Yao Xu, Yilu Liu
    Abstract:

    This work investigates the Data Quality Issue for synchrophasor applications, and pays particular attention to synchronization signal loss and synchrophasor Data loss events. First, the historical synchronization signal loss events are analyzed and the potential reasons and solutions are discussed. Then, the scenario of a small amount of synchrophasor Data loss is studied and a Lagrange interpolating polynomial method is used to adaptively estimate the incomplete and missing Data. The performance of proposed method is demonstrated with simulation results. Specifically, the proposed method considers the trade-off between the estimation accuracy and the hardware cost, and could be efficiently employed in reality.

Dao Zhou - One of the best experts on this subject based on the ideXlab platform.

  • Data Quality Issues for synchrophasor applications Part I: a review
    Journal of Modern Power Systems and Clean Energy, 2016
    Co-Authors: Can Huang, Zhuohong Pan, Jiahui Guo, Dao Zhou, Fangxing Li, Yong Liu, Yilu Liu
    Abstract:

    Synchrophasor systems, providing low-latency, high-precision, and time-synchronized measurements to enhance power grid performances, are deployed globally. However, the synchrophasor system as a physical network, involves communication constraints and Data Quality Issues, which will impact or even disable certain synchrophasor applications. This work investigates the Data Quality Issue for synchrophasor applications. In Part I, the standards of synchrophasor systems and the classifications and Data Quality requirements of synchrophasor applications are reviewed. Also, the actual events of synchronization signal accuracy, synchrophasor Data loss, and latency are counted and analyzed. The review and statistics are expected to provide an overall picture of Data accuracy, loss, and latency Issues for synchrophasor applications.

  • Data Quality Issues for synchrophasor applications Part II: problem formulation and potential solutions
    Journal of Modern Power Systems and Clean Energy, 2016
    Co-Authors: Can Huang, Lingwei Zhan, Qinran Hu, Dao Zhou, Fangxing Li, Yao Xu, Yilu Liu
    Abstract:

    This work investigates the Data Quality Issue for synchrophasor applications, and pays particular attention to synchronization signal loss and synchrophasor Data loss events. First, the historical synchronization signal loss events are analyzed and the potential reasons and solutions are discussed. Then, the scenario of a small amount of synchrophasor Data loss is studied and a Lagrange interpolating polynomial method is used to adaptively estimate the incomplete and missing Data. The performance of proposed method is demonstrated with simulation results. Specifically, the proposed method considers the trade-off between the estimation accuracy and the hardware cost, and could be efficiently employed in reality.

Fangxing Li - One of the best experts on this subject based on the ideXlab platform.

  • Data Quality Issues for synchrophasor applications Part I: a review
    Journal of Modern Power Systems and Clean Energy, 2016
    Co-Authors: Can Huang, Zhuohong Pan, Jiahui Guo, Dao Zhou, Fangxing Li, Yong Liu, Yilu Liu
    Abstract:

    Synchrophasor systems, providing low-latency, high-precision, and time-synchronized measurements to enhance power grid performances, are deployed globally. However, the synchrophasor system as a physical network, involves communication constraints and Data Quality Issues, which will impact or even disable certain synchrophasor applications. This work investigates the Data Quality Issue for synchrophasor applications. In Part I, the standards of synchrophasor systems and the classifications and Data Quality requirements of synchrophasor applications are reviewed. Also, the actual events of synchronization signal accuracy, synchrophasor Data loss, and latency are counted and analyzed. The review and statistics are expected to provide an overall picture of Data accuracy, loss, and latency Issues for synchrophasor applications.

  • Data Quality Issues for synchrophasor applications Part II: problem formulation and potential solutions
    Journal of Modern Power Systems and Clean Energy, 2016
    Co-Authors: Can Huang, Lingwei Zhan, Qinran Hu, Dao Zhou, Fangxing Li, Yao Xu, Yilu Liu
    Abstract:

    This work investigates the Data Quality Issue for synchrophasor applications, and pays particular attention to synchronization signal loss and synchrophasor Data loss events. First, the historical synchronization signal loss events are analyzed and the potential reasons and solutions are discussed. Then, the scenario of a small amount of synchrophasor Data loss is studied and a Lagrange interpolating polynomial method is used to adaptively estimate the incomplete and missing Data. The performance of proposed method is demonstrated with simulation results. Specifically, the proposed method considers the trade-off between the estimation accuracy and the hardware cost, and could be efficiently employed in reality.

Fatma-zohra Hannou - One of the best experts on this subject based on the ideXlab platform.

  • Explaining Query Answer Completeness and Correctness with Partition Patterns
    2019
    Co-Authors: Fatma-zohra Hannou, Bernd Amann, Mohamed-amine Baazizi
    Abstract:

    Information incompleteness is a major Data Quality Issue which is amplified by the increasing amount of Data collected from unreliable sources. Assessing the completeness of Data is crucial for determining the Quality of the Data itself, but also for verifying the validity of query answers over incomplete Data. In this article, we tackle the Issue of efficiently describing and inferring knowledge about Data completeness w.r.t. to a complete reference Data set and study the use of a partition pattern algebra for summarizing the completeness and validity of query answers. We describe an implementation and experiments with a real-world Dataset to validate the effectiveness and the efficiency of our approach.

  • A Pattern Model and Algebra for Representing and Querying Relative Information Completenes
    2019
    Co-Authors: Fatma-zohra Hannou
    Abstract:

    Information incompleteness is a major Data Quality Issue which is amplified by the increasing amount of Data collected from unreliable sources. Assessing the completeness of Data is crucial for determining the Quality of the Data and the validity of query answers.In this work, we tackle the Issue of extracting and reasoning about complete and missing information under relative information completeness setting. Under this setting, the completeness of a Dataset is assessed with respect to a complete reference Dataset. We advance the field by proposing two contributions: a pattern model for providing minimal covers summarizing the extent of complete and missing Data partitions and a pattern algebra for deriving minimal pattern covers for query answers to analyze their validity.The completeness pattern framework presents an intriguing opportunity to achieve many applications, particularly those aiming at improving the Quality of tasks impacted by missing Data. Data imputation is a well-known technique for repairing missing Data values but can incur a prohibitive cost when applied to large Data sets. Query-driven imputation offers a better alternative as it allows for We adopt a rule-based query rewriting technique for imputing the answers of aggregation queries that are missing or suffer from incorrectness due to Data incompleteness. We present a novel query rewriting mechanism that is guided by the completeness pattern model and algebra.We also investigate the generalization of our pattern model for summarizing any Data fragments. Summaries can be queried to analyze and compare Data fragments in a synthetic and flexible way.

  • Explaining Query Answer Completeness and Correctness with Minimal Pattern Covers
    2019
    Co-Authors: Fatma-zohra Hannou, Bernd Amann, Mohamed-amine Baazizi
    Abstract:

    Information incompleteness is a major Data Quality Issue which is amplified by the increasing amount of Data collected from unreliable sources. Assessing the completeness of Data is crucial for determining the Quality of the Data itself , but also for verifying the validity of query answers over incomplete Data. While there exists an important amount of work on modeling Data completeness, deriving this completeness information has not received much attention. In this work, we tackle the Issue of efficiently describing and inferring knowledge about Data completeness w.r.t. to a complete reference Data set and study the use of a pattern algebra for summarizing the completeness and validity of query answers. We describe an implementation and experiments with a real-world Dataset to validate the effectiveness and the efficiency of our approach.