The Experts below are selected from a list of 273 Experts worldwide ranked by ideXlab platform

Erik Ordentlich - One of the best experts on this subject based on the ideXlab platform.

Tsachy Weissman - One of the best experts on this subject based on the ideXlab platform.

  • Concentration inequalities for the Empirical Distribution of discrete Distributions: beyond the method of types
    Information and Inference: A Journal of the IMA, 2019
    Co-Authors: Jay Mardia, Jiantao Jiao, Ervin Tánczos, Robert Nowak, Tsachy Weissman
    Abstract:

    Abstract We study concentration inequalities for the Kullback–Leibler (KL) divergence between the Empirical Distribution and the true Distribution. Applying a recursion technique, we improve over the method of types bound uniformly in all regimes of sample size $n$ and alphabet size $k$, and the improvement becomes more significant when $k$ is large. We discuss the applications of our results in obtaining tighter concentration inequalities for $L_1$ deviations of the Empirical Distribution from the true Distribution, and the difference between concentration around the expectation or zero. We also obtain asymptotically tight bounds on the variance of the KL divergence between the Empirical and true Distribution, and demonstrate their quantitatively different behaviours between small and large sample sizes compared to the alphabet size.

  • Concentration Inequalities for the Empirical Distribution
    arXiv: Information Theory, 2018
    Co-Authors: Jay Mardia, Jiantao Jiao, Ervin Tánczos, Robert Nowak, Tsachy Weissman
    Abstract:

    We study concentration inequalities for the Kullback--Leibler (KL) divergence between the Empirical Distribution and the true Distribution. Applying a recursion technique, we improve over the method of types bound uniformly in all regimes of sample size $n$ and alphabet size $k$, and the improvement becomes more significant when $k$ is large. We discuss the applications of our results in obtaining tighter concentration inequalities for $L_1$ deviations of the Empirical Distribution from the true Distribution, and the difference between concentration around the expectation or zero. We also obtain asymptotically tight bounds on the variance of the KL divergence between the Empirical and true Distribution, and demonstrate their quantitatively different behaviors between small and large sample sizes compared to the alphabet size.

  • the Empirical Distribution of rate constrained source codes
    International Symposium on Information Theory, 2004
    Co-Authors: Tsachy Weissman, Erik Ordentlich
    Abstract:

    This paper describes the Empirical Distribution of rate-constrained source codes. The existence of good code sequences with Empirical Distributions achieves the minimum mutual information in the rate distortion theory. A source code has entropy of capacity achieving convergence in channel input Distributions.

  • Inequalities for the L1 Deviation of the Empirical Distribution
    2003
    Co-Authors: Tsachy Weissman, Sergio Verdu, Erik Ordentlich, Gadiel Seroussi, Marcelo Weinberger
    Abstract:

    We derive bounds on the probability that the L1 distance between the Empirical Distribution of a sequence of independent identically distributed random variables and the true Distribution is more than a specified value. We also derive a generalization of Pinsker’s inequality relating the L1 distance to the divergence.

  • ISIT - The Empirical Distribution of rate-constrained source codes
    International Symposium onInformation Theory 2004. ISIT 2004. Proceedings., 1
    Co-Authors: Tsachy Weissman, Erik Ordentlich
    Abstract:

    This paper describes the Empirical Distribution of rate-constrained source codes. The existence of good code sequences with Empirical Distributions achieves the minimum mutual information in the rate distortion theory. A source code has entropy of capacity achieving convergence in channel input Distributions.

Michał Pulit - One of the best experts on this subject based on the ideXlab platform.

  • Nonparametric estimation of the ROC curve based on smoothed Empirical Distribution functions
    Statistics and Computing, 2013
    Co-Authors: Alicja Jokiel-rokita, Michał Pulit
    Abstract:

    The receiver operating characteristic (ROC) curve is a graphical representation of the relationship between false positive and true positive rates. It is a widely used statistical tool for describing the accuracy of a diagnostic test. In this paper we propose a new nonparametric ROC curve estimator based on the smoothed Empirical Distribution functions. We prove its strong consistency and perform a simulation study to compare it with some other popular nonparametric estimators of the ROC curve. We also apply the proposed method to a real data set.

Shan Sun - One of the best experts on this subject based on the ideXlab platform.

  • Perturbed Empirical Distribution functions and quantiles under dependence
    Journal of Theoretical Probability, 1995
    Co-Authors: Shan Sun
    Abstract:

    In this note we consider the perturbed Empirical Distribution functions of the form\(\hat F_n (x) = 1/n \Sigma _{i = 1 }^{n } K_n (x - X_i ), x \in \mathbb{R}, n \underset{\raise0.3em\hbox{$\smash{\scriptscriptstyle-}$}}{ > } 1\), where {Xi,i>-1} is a sequence of strong mixing, nonstationary random variables\(K_n (x) = \smallint ^x _{ - \infty } k_n (t) dt, k_{n } (t) = \alpha _n^{ - 1} k(t\alpha ^{ - 1} )\), {an} is a sequence of positive real numbers such thatan→0 asn→∞ andk is a probability density function. We establish the necessary and sufficient conditions for the asymptotic normality of,\(\hat F_n (x)\). Similar results are also provided for the perturbed sample quantiles\(\hat \xi _n (p) = inf\{ x \varepsilon \mathbb{R}, \hat F_n (x) \geqslant p, 0< p< 1\} \), which is an inverse function of\(\hat F_n (x)\).

Madan L. Puri - One of the best experts on this subject based on the ideXlab platform.