The Experts below are selected from a list of 324 Experts worldwide ranked by ideXlab platform
Charalambos D. Charalambous - One of the best experts on this subject based on the ideXlab platform.
-
joint nonanticipative Rate Distortion Function for a tuple of random processes with individual fidelity criteria
arXiv: Information Theory, 2021Co-Authors: Charalambos D. Charalambous, Evagoras StylianouAbstract:The joint nonanticipative Rate Distortion Function (NRDF) for a tuple of random processes with individual fidelity criteria is considered. Structural properties of optimal test channel distributions are derived. Further, for the application example of the joint NRDF of a tuple of jointly multivariate Gaussian Markov processes with individual square-error fidelity criteria, a realization of the reproduction processes which induces the optimal test channel distribution is derived, and the corresponding joint NRDF is characterized. The analysis of the simplest example, of a tuple of scalar correlated Markov processes, illustRates many of the challenging aspects of such problems.
-
joint Rate Distortion Function of a tuple of correlated multivariate gaussian sources with individual fidelity criteria
arXiv: Information Theory, 2021Co-Authors: Evagoras Stylianou, Charalambos D. Charalambous, Themistoklis CharalambousAbstract:In this paper we analyze the joint Rate Distortion Function (RDF), for a tuple of correlated sources taking values in abstract alphabet spaces (i.e., continuous) subject to two individual Distortion criteria. First, we derive structural properties of the realizations of the reproduction Random Variables (RVs), which induce the corresponding optimal test channel distributions of the joint RDF. Second, we consider a tuple of correlated multivariate jointly Gaussian RVs, $X_1 : \Omega \rightarrow {\mathbb R}^{p_1}, X_2 : \Omega \rightarrow {\mathbb R}^{p_2}$ with two square-error fidelity criteria, and we derive additional structural properties of the optimal realizations, and use these to characterize the RDF as a convex optimization problem with respect to the parameters of the realizations. We show that the computation of the joint RDF can be performed by semidefinite programming. Further, we derive closed-form expressions of the joint RDF, such that Gray's [1] lower bounds hold with equality, and verify their consistency with the semidefinite programming computations.
-
Generalizations of Nonanticipative Rate Distortion Function to Multivariate Nonstationary Gaussian Autoregressive Processes
2019 IEEE 58th Conference on Decision and Control (CDC), 2019Co-Authors: Charalambos D. Charalambous, Themistoklis Charalambous, Christos Kourtellaris, Jan H. Van SchuppenAbstract:The characterizations of nonanticipative Rate Distortion Function (NRDF) on a finite horizon are generalized to nonstationary multivariate Gaussian order L autoregressive, AR(L), source processes, with respect to mean square error (MSE) Distortion Functions. It is shown that the optimal reproduction distributions are induced by a reproduction process, which is a linear Function of the state of the source, its best mean-square error estimate, and a Gaussian random process.
-
asymptotic reverse waterfilling characterization of nonanticipative Rate Distortion Function of vector valued gauss markov processes with mse Distortion
Conference on Decision and Control, 2018Co-Authors: Photios A Stavrou, Themistoklis Charalambous, Charalambos D. Charalambous, Sergey Loyka, Mikael SkoglundAbstract:We analyze the asymptotic nonanticipative Rate Distortion Function (NRDF) of vector-valued Gauss-Markov processes subject to a mean-squared error (MSE) Distortion Function. We derive a parametric characterization in terms of a reverse-waterfilling algorithm, that requires the solution of a matrix Riccati algebraic equation (RAE). Further, we develop an algorithm reminiscent of the classical reverse-waterfilling algorithm that provides an upper bound to the optimal solution of the reverse-waterfilling optimization problem, and under certain cases, it opeRates at the NRDF. Moreover, using the characterization of the reverse-waterfilling algorithm, we derive the analytical solution of the NRDF, for a simple two-dimensional parallel Gauss-Markov process. The efficacy of our proposed algorithm is demonstRated via an example.
-
optimal estimation via nonanticipative Rate Distortion Function and applications to time varying gauss markov processes
Siam Journal on Control and Optimization, 2018Co-Authors: Photios A Stavrou, Themistoklis Charalambous, Charalambos D. Charalambous, Sergey LoykaAbstract:In this paper, we develop finite-time horizon causal filters for general processes taking values in Polish spaces using the nonanticipative Rate Distortion Function ($NRDF$). Subsequently, we apply...
Photios A Stavrou - One of the best experts on this subject based on the ideXlab platform.
-
a novel sequential Rate Distortion Function to compute partially observable markov processes with mse Distortion
arXiv: Information Theory, 2019Co-Authors: Photios A Stavrou, Mikael SkoglundAbstract:We develop a new sequential Rate Distortion Function to compute lower bounds on the average length of all causal prefix free codes for partially observable multivariate Markov processes with mean-squared error Distortion constraint. Our information measure is characterized by a variant of causally conditioned directed information and it is utilized in various applications examples. First, it is used to optimally characterize a finite dimensional optimization problem for partially observable multivariate Gauss-Markov processes and to obtain the optimal linear policies that achieve the solution of this problem. Under the assumption that all matrices commute by pairs, we show that our problem can be cast as a convex optimization problem and achieves its lower Rates. We also derive sufficient conditions which ensure that our assumption holds. Then, we compute the optimization problem by solving the KKT conditions and deriving a non-trivial reverse-waterfilling algorithm that we also implement. If our assumption is not met, then, one can still use it to derive sub-optimal waterfilling solutions on the obtained finite dimensional optimization problem. For scalar-valued Gauss-Markov processes with additional observations noise, we derive a new closed form solution and we compared it with the analytical solution obtained for scalar-valued Gauss-Markov processes to infer about the Rate loss due to having the additional observations noise. For partially observable time-invariant Markov processes (without observations noise) driven by an additive i.i.d. non-Gaussian system's noise process, we recover using an alternative approach and thus strengthening a recent result by Kostina and Hassibi in [1, Theorem 9] whereas for time-invariant parallel and spatially identically distributed Markov processes driven by additive non-Gaussian noise process we also derive new analytical lower bounds.
-
asymptotic reverse waterfilling characterization of nonanticipative Rate Distortion Function of vector valued gauss markov processes with mse Distortion
Conference on Decision and Control, 2018Co-Authors: Photios A Stavrou, Themistoklis Charalambous, Charalambos D. Charalambous, Sergey Loyka, Mikael SkoglundAbstract:We analyze the asymptotic nonanticipative Rate Distortion Function (NRDF) of vector-valued Gauss-Markov processes subject to a mean-squared error (MSE) Distortion Function. We derive a parametric characterization in terms of a reverse-waterfilling algorithm, that requires the solution of a matrix Riccati algebraic equation (RAE). Further, we develop an algorithm reminiscent of the classical reverse-waterfilling algorithm that provides an upper bound to the optimal solution of the reverse-waterfilling optimization problem, and under certain cases, it opeRates at the NRDF. Moreover, using the characterization of the reverse-waterfilling algorithm, we derive the analytical solution of the NRDF, for a simple two-dimensional parallel Gauss-Markov process. The efficacy of our proposed algorithm is demonstRated via an example.
-
optimal estimation via nonanticipative Rate Distortion Function and applications to time varying gauss markov processes
Siam Journal on Control and Optimization, 2018Co-Authors: Photios A Stavrou, Themistoklis Charalambous, Charalambos D. Charalambous, Sergey LoykaAbstract:In this paper, we develop finite-time horizon causal filters for general processes taking values in Polish spaces using the nonanticipative Rate Distortion Function ($NRDF$). Subsequently, we apply...
-
finite time nonanticipative Rate Distortion Function for time varying scalar valued gauss markov sources
IEEE Control Systems Letters, 2018Co-Authors: Photios A Stavrou, Themistoklis Charalambous, Charalambos D. CharalambousAbstract:We derive the finite-time horizon nonanticipative Rate Distortion Function (NRDF) of time-varying scalar Gauss–Markov sources under an average mean squared-error (MSE) Distortion fidelity. Further, we show that a conditionally Gaussian reproduction process realizes the optimal reproduction distribution, and this is determined from the solution of a dynamic reverse-waterfilling optimization problem. We provide an iterative algorithm that approximates the solution of the dynamic reverse-waterfilling problem. From the above results, we also obtain, as a special case, the NRDF under a per-letter or pointwise MSE Distortion fidelity, and we draw connections to the classical RDF of Gaussian processes. Our results are corroboRated with illustrative examples.
-
information nonanticipative Rate Distortion Function and its applications
arXiv: Information Theory, 2015Co-Authors: Photios A Stavrou, Christos K Kourtellaris, Charalambos D. CharalambousAbstract:In this chapter, we introduce the information nonanticipative Rate Distortion Function (RDF), and we compare it with the classical information RDF, identifying certain limitations of the later, with respect to nonanticipative or real-time transmission for delay-sensitive applications. Then, we proceed further to describe applications of nonanticipative RDF in (1) joint source-channel coding (JSCC) using nonanticipative (delayless) transmission, and in (2) bounding the optimal performance theoretically attainable (OPTA) by noncausal and causal codes for general sources. Finally, to facilitate the application of the information nonanticipative RDF in computing the aforementioned bounds and in applying it to JSCC based on nonanticipative transmission, we proceed further to present the expression of the optimal reproduction distribution for nonstationary sources.
Themistoklis Charalambous - One of the best experts on this subject based on the ideXlab platform.
-
joint Rate Distortion Function of a tuple of correlated multivariate gaussian sources with individual fidelity criteria
arXiv: Information Theory, 2021Co-Authors: Evagoras Stylianou, Charalambos D. Charalambous, Themistoklis CharalambousAbstract:In this paper we analyze the joint Rate Distortion Function (RDF), for a tuple of correlated sources taking values in abstract alphabet spaces (i.e., continuous) subject to two individual Distortion criteria. First, we derive structural properties of the realizations of the reproduction Random Variables (RVs), which induce the corresponding optimal test channel distributions of the joint RDF. Second, we consider a tuple of correlated multivariate jointly Gaussian RVs, $X_1 : \Omega \rightarrow {\mathbb R}^{p_1}, X_2 : \Omega \rightarrow {\mathbb R}^{p_2}$ with two square-error fidelity criteria, and we derive additional structural properties of the optimal realizations, and use these to characterize the RDF as a convex optimization problem with respect to the parameters of the realizations. We show that the computation of the joint RDF can be performed by semidefinite programming. Further, we derive closed-form expressions of the joint RDF, such that Gray's [1] lower bounds hold with equality, and verify their consistency with the semidefinite programming computations.
-
Generalizations of Nonanticipative Rate Distortion Function to Multivariate Nonstationary Gaussian Autoregressive Processes
2019 IEEE 58th Conference on Decision and Control (CDC), 2019Co-Authors: Charalambos D. Charalambous, Themistoklis Charalambous, Christos Kourtellaris, Jan H. Van SchuppenAbstract:The characterizations of nonanticipative Rate Distortion Function (NRDF) on a finite horizon are generalized to nonstationary multivariate Gaussian order L autoregressive, AR(L), source processes, with respect to mean square error (MSE) Distortion Functions. It is shown that the optimal reproduction distributions are induced by a reproduction process, which is a linear Function of the state of the source, its best mean-square error estimate, and a Gaussian random process.
-
asymptotic reverse waterfilling characterization of nonanticipative Rate Distortion Function of vector valued gauss markov processes with mse Distortion
Conference on Decision and Control, 2018Co-Authors: Photios A Stavrou, Themistoklis Charalambous, Charalambos D. Charalambous, Sergey Loyka, Mikael SkoglundAbstract:We analyze the asymptotic nonanticipative Rate Distortion Function (NRDF) of vector-valued Gauss-Markov processes subject to a mean-squared error (MSE) Distortion Function. We derive a parametric characterization in terms of a reverse-waterfilling algorithm, that requires the solution of a matrix Riccati algebraic equation (RAE). Further, we develop an algorithm reminiscent of the classical reverse-waterfilling algorithm that provides an upper bound to the optimal solution of the reverse-waterfilling optimization problem, and under certain cases, it opeRates at the NRDF. Moreover, using the characterization of the reverse-waterfilling algorithm, we derive the analytical solution of the NRDF, for a simple two-dimensional parallel Gauss-Markov process. The efficacy of our proposed algorithm is demonstRated via an example.
-
optimal estimation via nonanticipative Rate Distortion Function and applications to time varying gauss markov processes
Siam Journal on Control and Optimization, 2018Co-Authors: Photios A Stavrou, Themistoklis Charalambous, Charalambos D. Charalambous, Sergey LoykaAbstract:In this paper, we develop finite-time horizon causal filters for general processes taking values in Polish spaces using the nonanticipative Rate Distortion Function ($NRDF$). Subsequently, we apply...
-
finite time nonanticipative Rate Distortion Function for time varying scalar valued gauss markov sources
IEEE Control Systems Letters, 2018Co-Authors: Photios A Stavrou, Themistoklis Charalambous, Charalambos D. CharalambousAbstract:We derive the finite-time horizon nonanticipative Rate Distortion Function (NRDF) of time-varying scalar Gauss–Markov sources under an average mean squared-error (MSE) Distortion fidelity. Further, we show that a conditionally Gaussian reproduction process realizes the optimal reproduction distribution, and this is determined from the solution of a dynamic reverse-waterfilling optimization problem. We provide an iterative algorithm that approximates the solution of the dynamic reverse-waterfilling problem. From the above results, we also obtain, as a special case, the NRDF under a per-letter or pointwise MSE Distortion fidelity, and we draw connections to the classical RDF of Gaussian processes. Our results are corroboRated with illustrative examples.
Jan Ostergaard - One of the best experts on this subject based on the ideXlab platform.
-
Stationarity in the Realizations of the Causal Rate-Distortion Function for One-Sided Stationary Sources
arXiv: Information Theory, 2018Co-Authors: Milan S. Derpich, Marco A. Guerrero, Jan OstergaardAbstract:This paper derives novel results on the characterization of the the causal information Rate-Distortion Function (IRDF) $R_{c}^{it}(D)$ for arbitrarily-distributed one-sided stationary $\kappa$-th order Markov source x(1),x(2),.... It is first shown that Gorbunov and Pinsker's results on the stationarity of the realizations to the causal IRDF (stated for two-sided stationary sources) do not apply to the commonly used family of asymptotic average single-letter (AASL) Distortion criteria. Moreover, we show that, in general, a reconstruction sequence cannot be both jointly stationary with a one-sided stationary source sequence and causally related to it. This implies that, in general, the causal IRDF for one-sided stationary sources cannot be realized by a stationary distribution. However, we prove that for an arbitrarily distributed one-sided stationary source and a large class of Distortion criteria (including AASL), the search for $R_{c}^{it}(D)$ can be restricted to distributions which yield the output sequence y(1), y(2),... jointly stationary with the source after $\kappa$ samples. Finally, we improve the definition of the stationary causal IRDF $\overline{R}_{c}^{it}(D)$ previously introduced by Derpich and {\O}stergaard for two-sided Markovian stationary sources and show that $\overline{R}_{c}^{it}(D)$ for a two-sided source ...,x(-1),x(0),x(1),... equals $R_{c}^{it}(D)$ for the associated one-sided source x(1), x(2),.... This implies that, for the Gaussian quadratic case, the practical zero-delay encoder-decoder pairs proposed by Derpich and {\O}stergaard for approaching $R_{c}^{it}(D)$ achieve an operational data Rate which exceeds $R_{c}^{it}(D)$ by less than $1+0.5 \log_2(2 \pi e /12) \simeq 1.254$ bits per sample.
-
improved upper bounds to the causal quadratic Rate Distortion Function for gaussian stationary sources
IEEE Transactions on Information Theory, 2012Co-Authors: Milan S. Derpich, Jan OstergaardAbstract:We improve the existing achievable Rate regions for causal and for zero-delay source coding of stationary Gaussian sources under an average mean squared error Distortion measure. To begin with, we find a closed-form expression for the information-theoretic causal Rate-Distortion Function (RDF) under such Distortion measure, denoted by Rcit(D), for first-order Gauss-Markov processes. Rcit(D) is a lower bound to the optimal performance theoretically attainable (OPTA) by any causal source code, namely Rcop(D). We show that, for Gaussian sources, the latter can also be upper bounded as Rcop(D) ≤ Rcit(D) + 0.5 log 2(2πe) bits/sample. In order to analyze Rcit(D) for arbitrary zero-mean Gaussian stationary sources, we introduce Rcit(D), the information-theoretic causal RDF when the reconstruction error is jointly stationary with the source. Based upon Rcit(D), we derive three closed-form upper bounds to the additive Rate loss defined as Rcit(D) - R(D), where R(D) denotes Shannon's RDF. Two of these bounds are strictly smaller than 0.5 bits/sample at all Rates. These bounds differ from one another in their tightness and ease of evaluation; the tighter the bound, the more involved its evaluation. We then show that, for any source spectral density and any positive Distortion D ≤ σx2, RU(D) can be realized by an additive white Gaussian noise channel surrounded by a unique set of causal pre-, post-, and feed- back niters. We show that finding such filters constitutes a convex optimization problem. In order to solve the latter, we propose an iterative optimization procedure that yields the optimal niters and is guaranteed to converge to Rcit(D). Finally, by establishing a connection to feedback quantization, we design a causal and a zero-delay coding scheme which, for Gaussian sources, achieves an operational Rate lower than Rcit(D) +0.254 and Rcit(D) + 0.754 bits/sample, respectively. This implies that the OPTA among all zero-delay source codes, denoted by Rzdop(D), is upper bounded as Rzdop(D) <; Rcit(D) + 1-254 <; R(D) + 1.754 bits/sample.
-
the high resolution Rate Distortion Function under the structural similarity index
EURASIP Journal on Advances in Signal Processing, 2011Co-Authors: Jan Ostergaard, Milan S. Derpich, Sumohana S ChannappayyaAbstract:We show that the structural similarity (SSIM) index, which is used in image processing to assess the similarity between an image representation and an original reference image, can be formulated as a locally quadratic Distortion measure. We, furthermore, show that recent results of Linder and Zamir on the Rate-Distortion Function (RDF) under locally quadratic Distortion measures are applicable to this SSIM Distortion measure. We finally derive the high-resolution SSIM-RDF and provide a simple method to numerically compute an approximation of the SSIM-RDF of real images.
-
improved upper bounds to the causal quadratic Rate Distortion Function for gaussian stationary sources
International Symposium on Information Theory, 2010Co-Authors: Milan S. Derpich, Jan OstergaardAbstract:We improve the existing achievable Rate regions for causal and for zero-delay source coding of stationary Gaussian sources for mean squared error (MSE) Distortion. First, we define the information-theoretic causal Rate-Distortion Function (RDF), Rit c (D). In order to analyze Rit c (D), we introduce Rit c (D), the information theoretic causal RDF when reconstruction error is jointly stationary with the source. Based upon Rit c (D), we derive four closed form upper bounds to the gap between Rit c (D) and Shannon's RDF, two of them strictly smaller than 0.5 bits/sample at all Rates. We then show that Rit c (D) can be realized by an AWGN channel surrounded by a unique set of causal pre-, post-, and feedback filters. We show that finding such filters constitutes a convex optimization problem and propose an iterative procedure to solve it. Finally, we build upon Rit c (D) to improve existing bounds on the optimal performance attainable by causal and zero-delay codes.
-
improved upper bounds to the causal quadratic Rate Distortion Function for gaussian stationary sources
arXiv: Information Theory, 2010Co-Authors: Milan S. Derpich, Jan OstergaardAbstract:We improve the existing achievable Rate regions for causal and for zero-delay source coding of stationary Gaussian sources under an average mean squared error (MSE) Distortion measure. To begin with, we find a closed-form expression for the information-theoretic causal Rate-Distortion Function (RDF) under such Distortion measure, denoted by $R_{c}^{it}(D)$, for first-order Gauss-Markov processes. Rc^{it}(D) is a lower bound to the optimal performance theoretically attainable (OPTA) by any causal source code, namely Rc^{op}(D). We show that, for Gaussian sources, the latter can also be upper bounded as Rc^{op}(D)\leq Rc^{it}(D) + 0.5 log_{2}(2\pi e) bits/sample. In order to analyze $R_{c}^{it}(D)$ for arbitrary zero-mean Gaussian stationary sources, we introduce \bar{Rc^{it}}(D), the information-theoretic causal RDF when the reconstruction error is jointly stationary with the source. Based upon \bar{Rc^{it}}(D), we derive three closed-form upper bounds to the additive Rate loss defined as \bar{Rc^{it}}(D) - R(D), where R(D) denotes Shannon's RDF. Two of these bounds are strictly smaller than 0.5 bits/sample at all Rates. These bounds differ from one another in their tightness and ease of evaluation; the tighter the bound, the more involved its evaluation. We then show that, for any source spectral density and any positive Distortion D\leq \sigma_{x}^{2}, \bar{Rc^{it}}(D) can be realized by an AWGN channel surrounded by a unique set of causal pre-, post-, and feedback filters. We show that finding such filters constitutes a convex optimization problem. In order to solve the latter, we propose an iterative optimization procedure that yields the optimal filters and is guaranteed to converge to \bar{Rc^{it}}(D). Finally, by establishing a connection to feedback quantization we design a causal and a zero-delay coding scheme which, for Gaussian sources, achieves...
Milan S. Derpich - One of the best experts on this subject based on the ideXlab platform.
-
Stationarity in the Realizations of the Causal Rate-Distortion Function for One-Sided Stationary Sources
arXiv: Information Theory, 2018Co-Authors: Milan S. Derpich, Marco A. Guerrero, Jan OstergaardAbstract:This paper derives novel results on the characterization of the the causal information Rate-Distortion Function (IRDF) $R_{c}^{it}(D)$ for arbitrarily-distributed one-sided stationary $\kappa$-th order Markov source x(1),x(2),.... It is first shown that Gorbunov and Pinsker's results on the stationarity of the realizations to the causal IRDF (stated for two-sided stationary sources) do not apply to the commonly used family of asymptotic average single-letter (AASL) Distortion criteria. Moreover, we show that, in general, a reconstruction sequence cannot be both jointly stationary with a one-sided stationary source sequence and causally related to it. This implies that, in general, the causal IRDF for one-sided stationary sources cannot be realized by a stationary distribution. However, we prove that for an arbitrarily distributed one-sided stationary source and a large class of Distortion criteria (including AASL), the search for $R_{c}^{it}(D)$ can be restricted to distributions which yield the output sequence y(1), y(2),... jointly stationary with the source after $\kappa$ samples. Finally, we improve the definition of the stationary causal IRDF $\overline{R}_{c}^{it}(D)$ previously introduced by Derpich and {\O}stergaard for two-sided Markovian stationary sources and show that $\overline{R}_{c}^{it}(D)$ for a two-sided source ...,x(-1),x(0),x(1),... equals $R_{c}^{it}(D)$ for the associated one-sided source x(1), x(2),.... This implies that, for the Gaussian quadratic case, the practical zero-delay encoder-decoder pairs proposed by Derpich and {\O}stergaard for approaching $R_{c}^{it}(D)$ achieve an operational data Rate which exceeds $R_{c}^{it}(D)$ by less than $1+0.5 \log_2(2 \pi e /12) \simeq 1.254$ bits per sample.
-
improved upper bounds to the causal quadratic Rate Distortion Function for gaussian stationary sources
IEEE Transactions on Information Theory, 2012Co-Authors: Milan S. Derpich, Jan OstergaardAbstract:We improve the existing achievable Rate regions for causal and for zero-delay source coding of stationary Gaussian sources under an average mean squared error Distortion measure. To begin with, we find a closed-form expression for the information-theoretic causal Rate-Distortion Function (RDF) under such Distortion measure, denoted by Rcit(D), for first-order Gauss-Markov processes. Rcit(D) is a lower bound to the optimal performance theoretically attainable (OPTA) by any causal source code, namely Rcop(D). We show that, for Gaussian sources, the latter can also be upper bounded as Rcop(D) ≤ Rcit(D) + 0.5 log 2(2πe) bits/sample. In order to analyze Rcit(D) for arbitrary zero-mean Gaussian stationary sources, we introduce Rcit(D), the information-theoretic causal RDF when the reconstruction error is jointly stationary with the source. Based upon Rcit(D), we derive three closed-form upper bounds to the additive Rate loss defined as Rcit(D) - R(D), where R(D) denotes Shannon's RDF. Two of these bounds are strictly smaller than 0.5 bits/sample at all Rates. These bounds differ from one another in their tightness and ease of evaluation; the tighter the bound, the more involved its evaluation. We then show that, for any source spectral density and any positive Distortion D ≤ σx2, RU(D) can be realized by an additive white Gaussian noise channel surrounded by a unique set of causal pre-, post-, and feed- back niters. We show that finding such filters constitutes a convex optimization problem. In order to solve the latter, we propose an iterative optimization procedure that yields the optimal niters and is guaranteed to converge to Rcit(D). Finally, by establishing a connection to feedback quantization, we design a causal and a zero-delay coding scheme which, for Gaussian sources, achieves an operational Rate lower than Rcit(D) +0.254 and Rcit(D) + 0.754 bits/sample, respectively. This implies that the OPTA among all zero-delay source codes, denoted by Rzdop(D), is upper bounded as Rzdop(D) <; Rcit(D) + 1-254 <; R(D) + 1.754 bits/sample.
-
the high resolution Rate Distortion Function under the structural similarity index
EURASIP Journal on Advances in Signal Processing, 2011Co-Authors: Jan Ostergaard, Milan S. Derpich, Sumohana S ChannappayyaAbstract:We show that the structural similarity (SSIM) index, which is used in image processing to assess the similarity between an image representation and an original reference image, can be formulated as a locally quadratic Distortion measure. We, furthermore, show that recent results of Linder and Zamir on the Rate-Distortion Function (RDF) under locally quadratic Distortion measures are applicable to this SSIM Distortion measure. We finally derive the high-resolution SSIM-RDF and provide a simple method to numerically compute an approximation of the SSIM-RDF of real images.
-
improved upper bounds to the causal quadratic Rate Distortion Function for gaussian stationary sources
International Symposium on Information Theory, 2010Co-Authors: Milan S. Derpich, Jan OstergaardAbstract:We improve the existing achievable Rate regions for causal and for zero-delay source coding of stationary Gaussian sources for mean squared error (MSE) Distortion. First, we define the information-theoretic causal Rate-Distortion Function (RDF), Rit c (D). In order to analyze Rit c (D), we introduce Rit c (D), the information theoretic causal RDF when reconstruction error is jointly stationary with the source. Based upon Rit c (D), we derive four closed form upper bounds to the gap between Rit c (D) and Shannon's RDF, two of them strictly smaller than 0.5 bits/sample at all Rates. We then show that Rit c (D) can be realized by an AWGN channel surrounded by a unique set of causal pre-, post-, and feedback filters. We show that finding such filters constitutes a convex optimization problem and propose an iterative procedure to solve it. Finally, we build upon Rit c (D) to improve existing bounds on the optimal performance attainable by causal and zero-delay codes.
-
improved upper bounds to the causal quadratic Rate Distortion Function for gaussian stationary sources
arXiv: Information Theory, 2010Co-Authors: Milan S. Derpich, Jan OstergaardAbstract:We improve the existing achievable Rate regions for causal and for zero-delay source coding of stationary Gaussian sources under an average mean squared error (MSE) Distortion measure. To begin with, we find a closed-form expression for the information-theoretic causal Rate-Distortion Function (RDF) under such Distortion measure, denoted by $R_{c}^{it}(D)$, for first-order Gauss-Markov processes. Rc^{it}(D) is a lower bound to the optimal performance theoretically attainable (OPTA) by any causal source code, namely Rc^{op}(D). We show that, for Gaussian sources, the latter can also be upper bounded as Rc^{op}(D)\leq Rc^{it}(D) + 0.5 log_{2}(2\pi e) bits/sample. In order to analyze $R_{c}^{it}(D)$ for arbitrary zero-mean Gaussian stationary sources, we introduce \bar{Rc^{it}}(D), the information-theoretic causal RDF when the reconstruction error is jointly stationary with the source. Based upon \bar{Rc^{it}}(D), we derive three closed-form upper bounds to the additive Rate loss defined as \bar{Rc^{it}}(D) - R(D), where R(D) denotes Shannon's RDF. Two of these bounds are strictly smaller than 0.5 bits/sample at all Rates. These bounds differ from one another in their tightness and ease of evaluation; the tighter the bound, the more involved its evaluation. We then show that, for any source spectral density and any positive Distortion D\leq \sigma_{x}^{2}, \bar{Rc^{it}}(D) can be realized by an AWGN channel surrounded by a unique set of causal pre-, post-, and feedback filters. We show that finding such filters constitutes a convex optimization problem. In order to solve the latter, we propose an iterative optimization procedure that yields the optimal filters and is guaranteed to converge to \bar{Rc^{it}}(D). Finally, by establishing a connection to feedback quantization we design a causal and a zero-delay coding scheme which, for Gaussian sources, achieves...
