Kullback-Leibler Divergence

14,000,000 Leading Edge Experts on the ideXlab platform

Scan Science and Technology

Contact Leading Edge Experts & Companies

Scan Science and Technology

Contact Leading Edge Experts & Companies

The Experts below are selected from a list of 15720 Experts worldwide ranked by ideXlab platform

Demba Diallo - One of the best experts on this subject based on the ideXlab platform.

  • statistical approach for nondestructive incipient crack detection and characterization using kullback leibler Divergence
    IEEE Transactions on Reliability, 2016
    Co-Authors: Jinane Harmouche, Claude Delpha, Demba Diallo, Yann Le Bihan
    Abstract:

    This paper is a contribution to the detection and characterisation of small cracks using Eddy Current Testing in the Non Destructive Evaluation framework. Small cracks are considered as incipient faults defined as gradual faults whose signature is weak and concealed by the noise. They are characterized by high signal to noise ratio and low fault to noise ratio. The detection and diagnosis of such faults is still an open challenge. For complex systems, model-based incipient fault detection and diagnosis (FDD) methods usually fail because of the inaccuracy of the model to describe all the phenomena and their interactions. Data-driven methods using statistical features are very promising as long as historical data are available. However in the case of incipient faults, there is not a significant variation of a single feature. The fault signature lies in the global variation of the signal properties. The proposed method relies on the Kullback-Leibler Divergence (KLD) as a nonparametric fault indicator. It measures the slight dissimilarities between the probability density functions of the current signal compared to the faultless or healthy one. Through experimental results, the KLD exhibits a higher sensitivity than the usual statistical features for the detection of small cracks (with dimensions in the order of 0.1 mm) realized in a nickel-based superalloy plate. Moreover, the detection is done with zero missed detection probability. Furthermore, the fault severity is assessed through the characteristics of the crack (surface, length, and depth). In the principal component analysis framework, the analysis of four statistical features (KLD, mean, variance, and maximum) dependency to the excitation frequency allows to discriminating among the cracks.

  • an optimal fault detection threshold for early detection using kullback leibler Divergence for unknown distribution data
    Signal Processing, 2016
    Co-Authors: Abdulrahman Youssef, Claude Delpha, Demba Diallo
    Abstract:

    The incipient fault detection in industrial processes with unknown distribution of measurements signals and unknown changed parameters is an important problem which has received much attention these last decades. However most of the detection methods (online and offline) need a priori knowledge on the signal distribution, changed parameters, and the change amplitude (Likelihood ratio test, Cusum, etc.). In this paper, an incipient fault detection method that does not need any a priori knowledge on the signals distribution or the changed parameters is proposed. This method is based on the analysis of the Kullback-Leibler Divergence (KLD) of probability distribution functions. However, the performance of the technique is highly dependent on the setting of a detection threshold and the environment noise level described through Signal to Noise Ratio (SNR) and Fault to Noise Ratio (FNR). In this paper, we develop an analytical model of the fault detection performances (False Alarm Probability and Missed Detection Probability). Thanks to this model, an optimisation procedure is applied to optimally set the fault detection threshold depending on the SNR and the fault severity. Compared to the usual settings, through simulation results and experimental data, the optimised threshold leads to higher efficiency for incipient fault detection in noisy environment. HighlightsWe propose an incipient fault detection method that does not need any a priori information on the signals distribution or the changed parameters.We show that the performance of the technique is highly dependent on the setting of a detection threshold and the environment noise level.We develop an analytical model of the fault detection performances (False Alarm Probability and Missed Detection Probability).Based on the aforementioned model, an optimisation procedure is applied to optimally set the fault detection threshold depending on the noise and the fault severity.Compared to the usual settings, a performed validation of this approach with through simulation results and experimental data is given.

  • incipient fault detection and diagnosis based on kullback leibler Divergence using principal component analysis
    Signal Processing, 2015
    Co-Authors: Jinane Harmouche, Claude Delpha, Demba Diallo
    Abstract:

    Most of fault indicators are devoted to detect deviations related to specific features but they fail to detect and estimate unpredictable slight distortions often caused by incipient faults. The Kullback-Leibler Divergence is characterised with a high sensitivity to incipient faults that cause unpredictable small changes in the process measurements. This work has two main objectives: first estimate the amplitude of incipient faults in multivariate processes based on the Divergence and second evaluate, through detection error probabilities, the performance of the Divergence in the detection of incipient faults in noisy environments.Throughout all the paper, the Fault-to-Noise Ratio (FNR) has been referred to as a comparative criterion between the fault level and noise; particularly the region around 0 dB of FNR is of interest in the evaluation. A theoretical study is developed to derive an analytical model of the Divergence that considers the presence of Gaussian noise and allows obtaining a theoretical estimate of the fault amplitude. After application on a simulated AR process, the fault amplitude estimate turns out to be an overestimation of the actual amplitude, therefore guaranteeing a safety margin for monitoring. Accurate fault severity estimation for an eddy currents application shows the effectiveness of this approach. HighlightsWe propose to enhance the fault detection approach based on the KLD modelling with the introduction of the noise.Based on the aforementioned model an estimator of the fault amplitude is developed and validated.The performances of the detection are studied in a noisy environment with the introduction of the Fault to Noise Ratio (FNR).The robustness of the proposed method is evaluated with the computation of the miss-detection and false alarms probabilities.A performed validation of this approach with a simulated AR model is given.

Jinane Harmouche - One of the best experts on this subject based on the ideXlab platform.

  • statistical approach for nondestructive incipient crack detection and characterization using kullback leibler Divergence
    IEEE Transactions on Reliability, 2016
    Co-Authors: Jinane Harmouche, Claude Delpha, Demba Diallo, Yann Le Bihan
    Abstract:

    This paper is a contribution to the detection and characterisation of small cracks using Eddy Current Testing in the Non Destructive Evaluation framework. Small cracks are considered as incipient faults defined as gradual faults whose signature is weak and concealed by the noise. They are characterized by high signal to noise ratio and low fault to noise ratio. The detection and diagnosis of such faults is still an open challenge. For complex systems, model-based incipient fault detection and diagnosis (FDD) methods usually fail because of the inaccuracy of the model to describe all the phenomena and their interactions. Data-driven methods using statistical features are very promising as long as historical data are available. However in the case of incipient faults, there is not a significant variation of a single feature. The fault signature lies in the global variation of the signal properties. The proposed method relies on the Kullback-Leibler Divergence (KLD) as a nonparametric fault indicator. It measures the slight dissimilarities between the probability density functions of the current signal compared to the faultless or healthy one. Through experimental results, the KLD exhibits a higher sensitivity than the usual statistical features for the detection of small cracks (with dimensions in the order of 0.1 mm) realized in a nickel-based superalloy plate. Moreover, the detection is done with zero missed detection probability. Furthermore, the fault severity is assessed through the characteristics of the crack (surface, length, and depth). In the principal component analysis framework, the analysis of four statistical features (KLD, mean, variance, and maximum) dependency to the excitation frequency allows to discriminating among the cracks.

  • incipient fault detection and diagnosis based on kullback leibler Divergence using principal component analysis
    Signal Processing, 2015
    Co-Authors: Jinane Harmouche, Claude Delpha, Demba Diallo
    Abstract:

    Most of fault indicators are devoted to detect deviations related to specific features but they fail to detect and estimate unpredictable slight distortions often caused by incipient faults. The Kullback-Leibler Divergence is characterised with a high sensitivity to incipient faults that cause unpredictable small changes in the process measurements. This work has two main objectives: first estimate the amplitude of incipient faults in multivariate processes based on the Divergence and second evaluate, through detection error probabilities, the performance of the Divergence in the detection of incipient faults in noisy environments.Throughout all the paper, the Fault-to-Noise Ratio (FNR) has been referred to as a comparative criterion between the fault level and noise; particularly the region around 0 dB of FNR is of interest in the evaluation. A theoretical study is developed to derive an analytical model of the Divergence that considers the presence of Gaussian noise and allows obtaining a theoretical estimate of the fault amplitude. After application on a simulated AR process, the fault amplitude estimate turns out to be an overestimation of the actual amplitude, therefore guaranteeing a safety margin for monitoring. Accurate fault severity estimation for an eddy currents application shows the effectiveness of this approach. HighlightsWe propose to enhance the fault detection approach based on the KLD modelling with the introduction of the noise.Based on the aforementioned model an estimator of the fault amplitude is developed and validated.The performances of the detection are studied in a noisy environment with the introduction of the Fault to Noise Ratio (FNR).The robustness of the proposed method is evaluated with the computation of the miss-detection and false alarms probabilities.A performed validation of this approach with a simulated AR model is given.

Claude Delpha - One of the best experts on this subject based on the ideXlab platform.

  • statistical approach for nondestructive incipient crack detection and characterization using kullback leibler Divergence
    IEEE Transactions on Reliability, 2016
    Co-Authors: Jinane Harmouche, Claude Delpha, Demba Diallo, Yann Le Bihan
    Abstract:

    This paper is a contribution to the detection and characterisation of small cracks using Eddy Current Testing in the Non Destructive Evaluation framework. Small cracks are considered as incipient faults defined as gradual faults whose signature is weak and concealed by the noise. They are characterized by high signal to noise ratio and low fault to noise ratio. The detection and diagnosis of such faults is still an open challenge. For complex systems, model-based incipient fault detection and diagnosis (FDD) methods usually fail because of the inaccuracy of the model to describe all the phenomena and their interactions. Data-driven methods using statistical features are very promising as long as historical data are available. However in the case of incipient faults, there is not a significant variation of a single feature. The fault signature lies in the global variation of the signal properties. The proposed method relies on the Kullback-Leibler Divergence (KLD) as a nonparametric fault indicator. It measures the slight dissimilarities between the probability density functions of the current signal compared to the faultless or healthy one. Through experimental results, the KLD exhibits a higher sensitivity than the usual statistical features for the detection of small cracks (with dimensions in the order of 0.1 mm) realized in a nickel-based superalloy plate. Moreover, the detection is done with zero missed detection probability. Furthermore, the fault severity is assessed through the characteristics of the crack (surface, length, and depth). In the principal component analysis framework, the analysis of four statistical features (KLD, mean, variance, and maximum) dependency to the excitation frequency allows to discriminating among the cracks.

  • an optimal fault detection threshold for early detection using kullback leibler Divergence for unknown distribution data
    Signal Processing, 2016
    Co-Authors: Abdulrahman Youssef, Claude Delpha, Demba Diallo
    Abstract:

    The incipient fault detection in industrial processes with unknown distribution of measurements signals and unknown changed parameters is an important problem which has received much attention these last decades. However most of the detection methods (online and offline) need a priori knowledge on the signal distribution, changed parameters, and the change amplitude (Likelihood ratio test, Cusum, etc.). In this paper, an incipient fault detection method that does not need any a priori knowledge on the signals distribution or the changed parameters is proposed. This method is based on the analysis of the Kullback-Leibler Divergence (KLD) of probability distribution functions. However, the performance of the technique is highly dependent on the setting of a detection threshold and the environment noise level described through Signal to Noise Ratio (SNR) and Fault to Noise Ratio (FNR). In this paper, we develop an analytical model of the fault detection performances (False Alarm Probability and Missed Detection Probability). Thanks to this model, an optimisation procedure is applied to optimally set the fault detection threshold depending on the SNR and the fault severity. Compared to the usual settings, through simulation results and experimental data, the optimised threshold leads to higher efficiency for incipient fault detection in noisy environment. HighlightsWe propose an incipient fault detection method that does not need any a priori information on the signals distribution or the changed parameters.We show that the performance of the technique is highly dependent on the setting of a detection threshold and the environment noise level.We develop an analytical model of the fault detection performances (False Alarm Probability and Missed Detection Probability).Based on the aforementioned model, an optimisation procedure is applied to optimally set the fault detection threshold depending on the noise and the fault severity.Compared to the usual settings, a performed validation of this approach with through simulation results and experimental data is given.

  • incipient fault detection and diagnosis based on kullback leibler Divergence using principal component analysis
    Signal Processing, 2015
    Co-Authors: Jinane Harmouche, Claude Delpha, Demba Diallo
    Abstract:

    Most of fault indicators are devoted to detect deviations related to specific features but they fail to detect and estimate unpredictable slight distortions often caused by incipient faults. The Kullback-Leibler Divergence is characterised with a high sensitivity to incipient faults that cause unpredictable small changes in the process measurements. This work has two main objectives: first estimate the amplitude of incipient faults in multivariate processes based on the Divergence and second evaluate, through detection error probabilities, the performance of the Divergence in the detection of incipient faults in noisy environments.Throughout all the paper, the Fault-to-Noise Ratio (FNR) has been referred to as a comparative criterion between the fault level and noise; particularly the region around 0 dB of FNR is of interest in the evaluation. A theoretical study is developed to derive an analytical model of the Divergence that considers the presence of Gaussian noise and allows obtaining a theoretical estimate of the fault amplitude. After application on a simulated AR process, the fault amplitude estimate turns out to be an overestimation of the actual amplitude, therefore guaranteeing a safety margin for monitoring. Accurate fault severity estimation for an eddy currents application shows the effectiveness of this approach. HighlightsWe propose to enhance the fault detection approach based on the KLD modelling with the introduction of the noise.Based on the aforementioned model an estimator of the fault amplitude is developed and validated.The performances of the detection are studied in a noisy environment with the introduction of the Fault to Noise Ratio (FNR).The robustness of the proposed method is evaluated with the computation of the miss-detection and false alarms probabilities.A performed validation of this approach with a simulated AR model is given.

Juan M R Parrondo - One of the best experts on this subject based on the ideXlab platform.

  • time series irreversibility a visibility graph approach
    European Physical Journal B, 2012
    Co-Authors: Lucas Lacasa, Edgar Roldan, Juan M R Parrondo, Angel M Nunez, Bartolo Luque
    Abstract:

    We propose a method to measure real-valued time series irreversibility which combines two different tools: the horizontal visibility algorithm and the Kullback-Leibler Divergence. This method maps a time series to a directed network according to a geometric criterion. The degree of irreversibility of the series is then estimated by the Kullback-Leibler Divergence (i.e. the distinguishability) between the inand outdegree distributions of the associated graph. The method is computationally efficient and does not require any ad hoc symbolization process. We find that the method correctly distinguishes between reversible and irreversible stationary time series, including analytical and numerical studies of its performance for: (i) reversible stochastic processes (uncorrelated and Gaussian linearly correlated), (ii) irreversible stochastic processes (a discrete flashing ratchet in an asymmetric potential), (iii) reversible (conservative) and irreversible (dissipative) chaotic maps, and (iv) dissipative chaotic maps in the presence of noise. Two alternative graph functionals, the degree and the degree-degree distributions, can be used as the Kullback-Leibler Divergence argument. The former is simpler and more intuitive and can be used as a benchmark, but in the case of an irreversible process with null net current, the degree-degree distribution has to be considered to identify the irreversible nature of the series.

  • entropy production and kullback leibler Divergence between stationary trajectories of discrete systems
    Physical Review E, 2012
    Co-Authors: Edgar Roldan, Juan M R Parrondo
    Abstract:

    The irreversibility of a stationary time series can be quantified using the Kullback-Leibler Divergence (KLD) between the probability of observing the series and the probability of observing the time-reversed series. Moreover, this KLD is a tool to estimate entropy production from stationary trajectories since it gives a lower bound to the entropy production of the physical process generating the series. In this paper we introduce analytical and numerical techniques to estimate the KLD between time series generated by several stochastic dynamics with a finite number of states. We examine the accuracy of our estimators for a specific example, a discrete flashing ratchet, and investigate how close the KLD is to the entropy production depending on the number of degrees of freedom of the system that are sampled in the trajectories.

  • time series irreversibility a visibility graph approach
    arXiv: Data Analysis Statistics and Probability, 2011
    Co-Authors: Lucas Lacasa, Edgar Roldan, Juan M R Parrondo, Angel M Nunez, Bartolo Luque
    Abstract:

    We propose a method to measure real-valued time series irreversibility which combines two differ- ent tools: the horizontal visibility algorithm and the Kullback-Leibler Divergence. This method maps a time series to a directed network according to a geometric criterion. The degree of irreversibility of the series is then estimated by the Kullback-Leibler Divergence (i.e. the distinguishability) between the in and out degree distributions of the associated graph. The method is computationally effi- cient, does not require any ad hoc symbolization process, and naturally takes into account multiple scales. We find that the method correctly distinguishes between reversible and irreversible station- ary time series, including analytical and numerical studies of its performance for: (i) reversible stochastic processes (uncorrelated and Gaussian linearly correlated), (ii) irreversible stochastic pro- cesses (a discrete flashing ratchet in an asymmetric potential), (iii) reversible (conservative) and irreversible (dissipative) chaotic maps, and (iv) dissipative chaotic maps in the presence of noise. Two alternative graph functionals, the degree and the degree-degree distributions, can be used as the Kullback-Leibler Divergence argument. The former is simpler and more intuitive and can be used as a benchmark, but in the case of an irreversible process with null net current, the degree-degree distribution has to be considered to identifiy the irreversible nature of the series.

Pengjian Shang - One of the best experts on this subject based on the ideXlab platform.

  • directed vector visibility graph from multivariate time series a new method to measure time series irreversibility
    Nonlinear Dynamics, 2021
    Co-Authors: Binbin Shang, Pengjian Shang
    Abstract:

    As a practical tool, visibility graph provides a different perspective to characterize time series. In this paper, we present a new visibility algorithm called directed vector visibility graph and combine it with the Kullback–Leibler Divergence to measure the irreversibility of multivariable time series. T directed vector visibility algorithm converts the time series into a directed network. Subsequently, the ingoing and outgoing degree distributions of the directed network can be got to calculate the Kullback–Leibler Divergence, which will be applied to assess the level of irreversibility of the time series. This is a simple and effective method without any special symbolic process. The numerical results from various types of systems are used to validate that this method can accurately distinguish reversible time series from those irreversible ones. Finally, we employ this method to estimate the irreversibility of financial time series and the results show that our method is efficient to analyze the financial time series irreversibility.

  • an improvement of the measurement of time series irreversibility with visibility graph approach
    Physica A-statistical Mechanics and Its Applications, 2018
    Co-Authors: Pengjian Shang, Hui Xiong
    Abstract:

    Abstract We propose a method to improve the measure of real-valued time series irreversibility which contains two tools: the directed horizontal visibility graph and the Kullback–Leibler Divergence. The degree of time irreversibility is estimated by the Kullback–Leibler Divergence between the i n and o u t degree distributions presented in the associated visibility graph. In our work, we reframe the i n and o u t degree distributions by encoding them with different embedded dimensions used in calculating permutation entropy(PE). With this improved method, we can not only estimate time series irreversibility efficiently, but also detect time series irreversibility from multiple dimensions. We verify the validity of our method and then estimate the amount of time irreversibility of series generated by chaotic maps as well as global stock markets over the period 2005–2015. The result shows that the amount of time irreversibility reaches the peak with embedded dimension d = 3 under circumstances of experiment and financial markets.