The Experts below are selected from a list of 540 Experts worldwide ranked by ideXlab platform

Massimo Franceschetti - One of the best experts on this subject based on the ideXlab platform.

  • Towards a Non-Stochastic Information Theory
    2019 IEEE International Symposium on Information Theory (ISIT), 2019
    Co-Authors: Anshuka Rangi, Massimo Franceschetti
    Abstract:

    The δ-mutual information between uncertain variables is introduced as a generalization of Nair's non-stochastic information functional. Several properties of this new quantity are illustrated, and used to prove a Channel Coding Theorem in a non-stochastic setting. Namely, it is shown that the largest δ mutual information between a metric space and its ε-packing equals the (ε,δ)-capacity of the space. This notion of capacity generalizes the Kolmogorov ε-capacity to packing sets of overlap at most δ, and is a variation of a previous definition proposed by one of the authors. These results provide a framework for developing a non-stochastic information theory motivated by potential applications in control and learning theories. Compared to previous non-stochastic approaches, the theory admits the possibility of deCoding errors as in Shannon's probabilistic setting, while retaining its worst-case non-stochastic character.

  • ISIT - Towards a Non-Stochastic Information Theory
    2019 IEEE International Symposium on Information Theory (ISIT), 2019
    Co-Authors: Anshuka Rangi, Massimo Franceschetti
    Abstract:

    The δ-mutual information between uncertain variables is introduced as a generalization of Nair’s non-stochastic information functional. Several properties of this new quantity are illustrated, and used to prove a Channel Coding Theorem in a non-stochastic setting. Namely, it is shown that the largest δ mutual information between a metric space and its ϵ-packing equals the (ϵ,δ)-capacity of the space. This notion of capacity generalizes the Kolmogorov ϵ -capacity to packing sets of overlap at most δ, and is a variation of a previous definition proposed by one of the authors. These results provide a framework for developing a non-stochastic information theory motivated by potential applications in control and learning theories. Compared to previous non-stochastic approaches, the theory admits the possibility of deCoding errors as in Shannon’s probabilistic setting, while retaining its worst-case non-stochastic character.

  • Towards a Non-Stochastic Information Theory.
    arXiv: Information Theory, 2019
    Co-Authors: Anshuka Rangi, Massimo Franceschetti
    Abstract:

    The $\delta$-mutual information between uncertain variables is introduced as a generalization of Nair's non-stochastic information functional. Several properties of this new quantity are illustrated, and used to prove a Channel Coding Theorem in a non-stochastic setting. Namely, it is shown that the largest $\delta$-mutual information between a metric space and its $\epsilon$-packing equals the $(\epsilon, \delta)$-capacity of the space. This notion of capacity generalizes the Kolmogorov $\epsilon$-capacity to packing sets of overlap at most $\delta$, and is a variation of a previous definition proposed by one of the authors. These results provide a framework for developing a non-stochastic information theory motivated by potential applications in control and learning theories. Compared to previous non-stochastic approaches, the theory admits the possibility of deCoding errors as in Shannon's probabilistic setting, while retaining its worst-case non-stochastic character.

Todd P. Coleman - One of the best experts on this subject based on the ideXlab platform.

  • ISIT - A stochastic control viewpoint on ‘Posterior Matching’-style feedback communication schemes
    2009 IEEE International Symposium on Information Theory, 2009
    Co-Authors: Todd P. Coleman
    Abstract:

    This paper re-visits Shayevitz & Feder's recent ‘Posterior Matching Scheme’, a deterministic, recursive, capacity-achieving feedback enCoding scheme for memoryless Channels. We here consider the feedback encoder design problem from a stochastic control perspective. The state of the system is the posterior distribution of the message given current outputs of the Channel. The per-trial reward is the average ‘reduction in distance’ of the posterior to the target unit step function. We show that the converse to the Channel Coding Theorem with feedback upper bounds the optimal reward, and that the posterior matching scheme is an optimal policy. We illustrate that this ‘reduction in distance’ symbolism leads to the existence of a Lyapunov function on the Markov chain under this optimal policy, which leads to demonstration of achievability for all rates less than capacity.

  • A stochastic control viewpoint on ‘Posterior Matching’-style feedback communication schemes
    2009 IEEE International Symposium on Information Theory, 2009
    Co-Authors: Todd P. Coleman
    Abstract:

    This paper re-visits Shayevitz & Feder's recent dasiaPosterior Matching Schemepsila, a deterministic, recursive, capacity-achieving feedback enCoding scheme for memoryless Channels. We here consider the feedback encoder design problem from a stochastic control perspective. The state of the system is the posterior distribution of the message given current outputs of the Channel. The per-trial reward is the average dasiareduction in distancepsila of the posterior to the target unit step function. We show that the converse to the Channel Coding Theorem with feedback upper bounds the optimal reward, and that the posterior matching scheme is an optimal policy. We illustrate that this dasiareduction in distancepsila symbolism leads to the existence of a Lyapunov function on the Markov chain under this optimal policy, which leads to demonstration of achievability for all rates less than capacity.

Ping Wang - One of the best experts on this subject based on the ideXlab platform.

  • Robust image hashing based on low-rank and sparse decomposition
    2016 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2016
    Co-Authors: Yuenan Li, Ping Wang
    Abstract:

    We propose in this paper a low-rank and sparse decomposition based image hashing algorithm, aiming to summarize the structural information and sparse salient components of digital image to compact digest. More specifically, we leverage compressive sampling and random projection to separately aggregate the low-rank approximation of input image and the spatial layout of salient components into binary hash. Owing to its capability of capturing and fusing intrinsic visual characteristics, the proposed work demonstrates high robustness and discriminability. As observed in content identification experiments, it shows much higher accuracy than state-of-the-art algorithms. Furthermore, we also analytically evaluate the security of the proposed hashing algorithm using the entropy based metric, and its performance in content identification is analyzed using the Channel Coding Theorem.

  • ICASSP - Robust image hashing based on low-rank and sparse decomposition
    2016 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), 2016
    Co-Authors: Yuenan Li, Ping Wang
    Abstract:

    We propose in this paper a low-rank and sparse decomposition based image hashing algorithm, aiming to summarize the structural information and sparse salient components of digital image to compact digest. More specifically, we leverage compressive sampling and random projection to separately aggregate the low-rank approximation of input image and the spatial layout of salient components into binary hash. Owing to its capability of capturing and fusing intrinsic visual characteristics, the proposed work demonstrates high robustness and discriminability. As observed in content identification experiments, it shows much higher accuracy than state-of-the-art algorithms. Furthermore, we also analytically evaluate the security of the proposed hashing algorithm using the entropy based metric, and its performance in content identification is analyzed using the Channel Coding Theorem.

H. Nagaoka - One of the best experts on this subject based on the ideXlab platform.

Nilanjana Datta - One of the best experts on this subject based on the ideXlab platform.

  • Smooth Entropies and the Quantum Information
    2020
    Co-Authors: Nilanjana Datta, Renato Renner
    Abstract:

    Many of the traditional results in information theory, such as the Channel Coding Theorem or the source Coding Theorem, are restricted to scenarios where the underlying resources are in- dependent and identically distributed (i.i.d.) over a large number of uses. To overcome this limitation, two different techniques, the informationspectrummethodandthesmoothentropyframework, have been developed independently. They are based on new en- tropymeasures, calledspectralentropy ratesandsmooth entropies, respectively,that generalizeShannon entropy (in the classicalcase) and von Neumann entropy (in the more general quantum case). Here, we show that the two techniques are closely related. More precisely, the spectral entropy rate can be seen as the asymptotic limit of the smooth entropy. Our results apply to the quantum set- ting and thus include the classical setting as a special case.

  • Smooth Entropies and the Quantum Information Spectrum
    IEEE Transactions on Information Theory, 2009
    Co-Authors: Nilanjana Datta, Renato Renner
    Abstract:

    Many of the traditional results in information theory, such as the Channel Coding Theorem or the source Coding Theorem, are restricted to scenarios where the underlying resources are independent and identically distributed (i.i.d.) over a large number of uses. To overcome this limitation, two different techniques, the information spectrum method and the smooth entropy framework, have been developed independently. They are based on new entropy measures, called spectral entropy rates and smooth entropies, respectively, that generalize Shannon entropy (in the classical case) and von Neumann entropy (in the more general quantum case). Here, we show that the two techniques are closely related. More precisely, the spectral entropy rate can be seen as the asymptotic limit of the smooth entropy. Our results apply to the quantum setting and thus include the classical setting as a special case.

  • A quantum version of Feinstein's Theorem and its application to Channel Coding
    2006 IEEE International Symposium on Information Theory, 2006
    Co-Authors: Nilanjana Datta, Tony Dorlas
    Abstract:

    In this paper, a quantum version of Feinstein's Theorem is developed. This is then used to give a completely self-contained proof of the direct Channel Coding Theorem, for transmission of classical information through a quantum Channel with Markovian correlated noise. Our proof does not rely on the Holevo-Schumacher-Westmoreland (HSW) Theorem. In addition, for the case of memoryless Channels, our method yields an alternative proof of the HSW Theorem

  • ISIT - A quantum version of Feinstein's Theorem and its application to Channel Coding
    2006 IEEE International Symposium on Information Theory, 2006
    Co-Authors: Nilanjana Datta, Tony Dorlas
    Abstract:

    In this paper, a quantum version of Feinstein's Theorem is developed. This is then used to give a completely self-contained proof of the direct Channel Coding Theorem, for transmission of classical information through a quantum Channel with Markovian correlated noise. Our proof does not rely on the Holevo-Schumacher-Westmoreland (HSW) Theorem. In addition, for the case of memoryless Channels, our method yields an alternative proof of the HSW Theorem.