Joint Entropy

14,000,000 Leading Edge Experts on the ideXlab platform

Scan Science and Technology

Contact Leading Edge Experts & Companies

Scan Science and Technology

Contact Leading Edge Experts & Companies

The Experts below are selected from a list of 9027 Experts worldwide ranked by ideXlab platform

Arman Rahmim - One of the best experts on this subject based on the ideXlab platform.

  • anatomy assisted pet image reconstruction incorporating multi resolution Joint Entropy
    Physics in Medicine and Biology, 2015
    Co-Authors: Jing Tang, Arman Rahmim
    Abstract:

    A promising approach in PET image reconstruction is to incorporate high resolution anatomical information (measured from MR or CT) taking the anato-functional similarity measures such as mutual information or Joint Entropy (JE) as the prior. These similarity measures only classify voxels based on intensity values, while neglecting structural spatial information. In this work, we developed an anatomy-assisted maximum a posteriori (MAP) reconstruction algorithm wherein the JE measure is supplied by spatial information generated using wavelet multi-resolution analysis. The proposed wavelet-based JE (WJE) MAP algorithm involves calculation of derivatives of the subband JE measures with respect to individual PET image voxel intensities, which we have shown can be computed very similarly to how the inverse wavelet transform is implemented. We performed a simulation study with the BrainWeb phantom creating PET data corresponding to different noise levels. Realistically simulated T1-weighted MR images provided by BrainWeb modeling were applied in the anatomy-assisted reconstruction with the WJE-MAP algorithm and the intensity-only JE-MAP algorithm. Quantitative analysis showed that the WJE-MAP algorithm performed similarly to the JE-MAP algorithm at low noise level in the gray matter (GM) and white matter (WM) regions in terms of noise versus bias tradeoff. When noise increased to medium level in the simulated data, the WJE-MAP algorithm started to surpass the JE-MAP algorithm in the GM region, which is less uniform with smaller isolated structures compared to the WM region. In the high noise level simulation, the WJE-MAP algorithm presented clear improvement over the JE-MAP algorithm in both the GM and WM regions. In addition to the simulation study, we applied the reconstruction algorithms to real patient studies involving DPA-173 PET data and Florbetapir PET data with corresponding T1-MPRAGE MRI images. Compared to the intensity-only JE-MAP algorithm, the WJE-MAP algorithm resulted in comparable regional mean values to those from the maximum likelihood algorithm while reducing noise. Achieving robust performance in various noise-level simulation and patient studies, the WJE-MAP algorithm demonstrates its potential in clinical quantitative PET imaging.

  • direct 4d reconstruction of parametric images incorporating anato functional Joint Entropy
    Physics in Medicine and Biology, 2010
    Co-Authors: Jing Tang, Hiroto Kuwabara, Dean F Wong, Arman Rahmim
    Abstract:

    We developed an anatomy-guided 4D closed-form algorithm to directly reconstruct parametric images from projection data for (nearly) irreversible tracers. Conventional methods consist of individually reconstructing 2D/3D PET data, followed by graphical analysis on the sequence of reconstructed image frames. The proposed direct reconstruction approach maintains the simplicity and accuracy of the expectation-maximization (EM) algorithm by extending the system matrix to include the relation between the parametric images and the measured data. A closed-form solution was achieved using a different hidden complete-data formulation within the EM framework. Furthermore, the proposed method was extended to maximum a posterior reconstruction via incorporation of MR image information, taking the Joint Entropy between MR and parametric PET features as the prior. Using realistic simulated noisy [11C]-naltrindole PET and MR brain images/data, the quantitative performance of the proposed methods was investigated. Significant improvements in terms of noise versus bias performance were demonstrated when performing direct parametric reconstruction, and additionally upon extending the algorithm to its Bayesian counterpart using the MR-PET Joint Entropy measure.

  • bayesian pet image reconstruction incorporating anato functional Joint Entropy
    Physics in Medicine and Biology, 2009
    Co-Authors: Jing Tang, Arman Rahmim
    Abstract:

    We developed a maximum a posterior (MAP) reconstruction method for positron emission tomography (PET) image reconstruction incorporating magnetic resonance (MR) image information, with the Joint Entropy between the PET and MR image features serving as the regularization constraint. A non-parametric method was used to estimate the Joint probability density of the PET and MR images. Using realistically simulated PET and MR human brain phantoms, the quantitative performance of the proposed algorithm was investigated. Incorporation of the anatomic information via this technique, after parameter optimization, was seen to dramatically improve the noise versus bias tradeoff in every region of interest, compared to the result from using conventional MAP reconstruction. In particular, hot lesions in the FDG PET image, which had no anatomical correspondence in the MR image, also had improved contrast versus noise tradeoff.Corrections were made to figures 3, 4 and 6, and to the second paragraph of section 3.1 on 13 November 2009. The corrected electronic version is identical to the print version.

  • direct 4d reconstruction of parametric images incorporating anato functional Joint Entropy
    IEEE Nuclear Science Symposium, 2008
    Co-Authors: Jing Tang, Hiroto Kuwabara, Dean F Wong, Arman Rahmim
    Abstract:

    We developed a closed-form 4D algorithm to directly reconstruct parametric images as obtained using the Patlak graphical method for (nearly) irreversible tracers. Conventional methods consist of individually reconstructing 2D/3D PET data, followed by graphical analysis on the sequence of reconstructed images. The proposed approach maintains the simplicity and accuracy of the EM algorithm by extending the system matrix to include the relation between the parametric images and the measured data. The proposed technique achieves a closed-form solution by utilizing a different hidden complete-data formulation within the EM framework. Additionally, the method is extended to maximum a posterior (MAP) reconstruction via incorporating MR image information, with the Joint Entropy between the MR and parametric PET features. A Parzen window method was used to estimate the Joint probability density of the MR and parametric PET images. Using realistic simulated [11C]-Naltrindole PET and MR brain images/data, the quantitative performance of the proposed methods was investigated. Significant improvements in terms of noise vs. bias performance have been achieved, when performing direct parametric reconstruction, and additionally when extending the algorithm to its Bayesian counter-part using MR-PET join Entropy.

  • Bayesian PET image reconstruction incorporating anato-functional Joint Entropy
    2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2008
    Co-Authors: Jing Tang, Benjamin M. W. Tsui, Arman Rahmim
    Abstract:

    We developed a maximum a posterior (MAP) reconstruction method for PET image reconstruction incorporating MR image information, with the Joint Entropy between the PET and MR image features serving as the prior. A non-parametric method was used to estimate the Joint probability density (JPD) of the PET and MR images. The sampling rate for Parzen window estimation of the JPD was studied for both simulated phantom and clinical FDG PET brain images. Using realistic simulated PET and MR brain phantoms, the quantitative performance of the proposed algorithm was investigated. In particular, variations in the weighting factor on the MAP prior as well as the variance in the Parzen window were examined. Incorporation of the anatomical information via this technique was seen to noticeably improve the noise vs. bias tradeoff in various regions of interest.

Jing Tang - One of the best experts on this subject based on the ideXlab platform.

  • anatomy assisted pet image reconstruction incorporating multi resolution Joint Entropy
    Physics in Medicine and Biology, 2015
    Co-Authors: Jing Tang, Arman Rahmim
    Abstract:

    A promising approach in PET image reconstruction is to incorporate high resolution anatomical information (measured from MR or CT) taking the anato-functional similarity measures such as mutual information or Joint Entropy (JE) as the prior. These similarity measures only classify voxels based on intensity values, while neglecting structural spatial information. In this work, we developed an anatomy-assisted maximum a posteriori (MAP) reconstruction algorithm wherein the JE measure is supplied by spatial information generated using wavelet multi-resolution analysis. The proposed wavelet-based JE (WJE) MAP algorithm involves calculation of derivatives of the subband JE measures with respect to individual PET image voxel intensities, which we have shown can be computed very similarly to how the inverse wavelet transform is implemented. We performed a simulation study with the BrainWeb phantom creating PET data corresponding to different noise levels. Realistically simulated T1-weighted MR images provided by BrainWeb modeling were applied in the anatomy-assisted reconstruction with the WJE-MAP algorithm and the intensity-only JE-MAP algorithm. Quantitative analysis showed that the WJE-MAP algorithm performed similarly to the JE-MAP algorithm at low noise level in the gray matter (GM) and white matter (WM) regions in terms of noise versus bias tradeoff. When noise increased to medium level in the simulated data, the WJE-MAP algorithm started to surpass the JE-MAP algorithm in the GM region, which is less uniform with smaller isolated structures compared to the WM region. In the high noise level simulation, the WJE-MAP algorithm presented clear improvement over the JE-MAP algorithm in both the GM and WM regions. In addition to the simulation study, we applied the reconstruction algorithms to real patient studies involving DPA-173 PET data and Florbetapir PET data with corresponding T1-MPRAGE MRI images. Compared to the intensity-only JE-MAP algorithm, the WJE-MAP algorithm resulted in comparable regional mean values to those from the maximum likelihood algorithm while reducing noise. Achieving robust performance in various noise-level simulation and patient studies, the WJE-MAP algorithm demonstrates its potential in clinical quantitative PET imaging.

  • direct 4d reconstruction of parametric images incorporating anato functional Joint Entropy
    Physics in Medicine and Biology, 2010
    Co-Authors: Jing Tang, Hiroto Kuwabara, Dean F Wong, Arman Rahmim
    Abstract:

    We developed an anatomy-guided 4D closed-form algorithm to directly reconstruct parametric images from projection data for (nearly) irreversible tracers. Conventional methods consist of individually reconstructing 2D/3D PET data, followed by graphical analysis on the sequence of reconstructed image frames. The proposed direct reconstruction approach maintains the simplicity and accuracy of the expectation-maximization (EM) algorithm by extending the system matrix to include the relation between the parametric images and the measured data. A closed-form solution was achieved using a different hidden complete-data formulation within the EM framework. Furthermore, the proposed method was extended to maximum a posterior reconstruction via incorporation of MR image information, taking the Joint Entropy between MR and parametric PET features as the prior. Using realistic simulated noisy [11C]-naltrindole PET and MR brain images/data, the quantitative performance of the proposed methods was investigated. Significant improvements in terms of noise versus bias performance were demonstrated when performing direct parametric reconstruction, and additionally upon extending the algorithm to its Bayesian counterpart using the MR-PET Joint Entropy measure.

  • bayesian pet image reconstruction incorporating anato functional Joint Entropy
    Physics in Medicine and Biology, 2009
    Co-Authors: Jing Tang, Arman Rahmim
    Abstract:

    We developed a maximum a posterior (MAP) reconstruction method for positron emission tomography (PET) image reconstruction incorporating magnetic resonance (MR) image information, with the Joint Entropy between the PET and MR image features serving as the regularization constraint. A non-parametric method was used to estimate the Joint probability density of the PET and MR images. Using realistically simulated PET and MR human brain phantoms, the quantitative performance of the proposed algorithm was investigated. Incorporation of the anatomic information via this technique, after parameter optimization, was seen to dramatically improve the noise versus bias tradeoff in every region of interest, compared to the result from using conventional MAP reconstruction. In particular, hot lesions in the FDG PET image, which had no anatomical correspondence in the MR image, also had improved contrast versus noise tradeoff.Corrections were made to figures 3, 4 and 6, and to the second paragraph of section 3.1 on 13 November 2009. The corrected electronic version is identical to the print version.

  • direct 4d reconstruction of parametric images incorporating anato functional Joint Entropy
    IEEE Nuclear Science Symposium, 2008
    Co-Authors: Jing Tang, Hiroto Kuwabara, Dean F Wong, Arman Rahmim
    Abstract:

    We developed a closed-form 4D algorithm to directly reconstruct parametric images as obtained using the Patlak graphical method for (nearly) irreversible tracers. Conventional methods consist of individually reconstructing 2D/3D PET data, followed by graphical analysis on the sequence of reconstructed images. The proposed approach maintains the simplicity and accuracy of the EM algorithm by extending the system matrix to include the relation between the parametric images and the measured data. The proposed technique achieves a closed-form solution by utilizing a different hidden complete-data formulation within the EM framework. Additionally, the method is extended to maximum a posterior (MAP) reconstruction via incorporating MR image information, with the Joint Entropy between the MR and parametric PET features. A Parzen window method was used to estimate the Joint probability density of the MR and parametric PET images. Using realistic simulated [11C]-Naltrindole PET and MR brain images/data, the quantitative performance of the proposed methods was investigated. Significant improvements in terms of noise vs. bias performance have been achieved, when performing direct parametric reconstruction, and additionally when extending the algorithm to its Bayesian counter-part using MR-PET join Entropy.

  • Bayesian PET image reconstruction incorporating anato-functional Joint Entropy
    2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro, 2008
    Co-Authors: Jing Tang, Benjamin M. W. Tsui, Arman Rahmim
    Abstract:

    We developed a maximum a posterior (MAP) reconstruction method for PET image reconstruction incorporating MR image information, with the Joint Entropy between the PET and MR image features serving as the prior. A non-parametric method was used to estimate the Joint probability density (JPD) of the PET and MR images. The sampling rate for Parzen window estimation of the JPD was studied for both simulated phantom and clinical FDG PET brain images. Using realistic simulated PET and MR brain phantoms, the quantitative performance of the proposed algorithm was investigated. In particular, variations in the weighting factor on the MAP prior as well as the variance in the Parzen window were examined. Incorporation of the anatomical information via this technique was seen to noticeably improve the noise vs. bias tradeoff in various regions of interest.

Ross T Whitaker - One of the best experts on this subject based on the ideXlab platform.

  • unsupervised information theoretic adaptive image filtering for image restoration
    IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006
    Co-Authors: Suyash P Awate, Ross T Whitaker
    Abstract:

    Image restoration is an important and widely studied problem in computer vision and image processing. Various image filtering strategies have been effective, but invariably make strong assumptions about the properties of the signal and/or degradation. Hence, these methods lack the generality to be easily applied to new applications or diverse image collections. This paper describes a novel unsupervised, information-theoretic, adaptive filter (UINTA) that improves the predictability of pixel intensities from their neighborhoods by decreasing their Joint Entropy. In this way, UINTA automatically discovers the statistical properties of the signal and can thereby restore a wide spectrum of images. The paper describes the formulation to minimize the Joint Entropy measure and presents several important practical considerations in estimating neighborhood statistics. It presents a series of results on both real and synthetic data along with comparisons with state-of-the-art techniques, including novel applications to medical image processing.

  • higher order image statistics for unsupervised information theoretic adaptive image filtering
    Computer Vision and Pattern Recognition, 2005
    Co-Authors: Suyash P Awate, Ross T Whitaker
    Abstract:

    The restoration of images is an important and widely studied problem in computer vision and image processing. Various image filtering strategies have been effective, but invariably make strong assumptions about the properties of the signal and/or degradation. Therefore, these methods typically lack the generality to be easily applied to new applications or diverse image collections. This paper describes a novel unsupervised, information-theoretic, adaptive filter (UINTA) that improves the predictability of pixel intensities from their neighborhoods by decreasing the Joint Entropy between them. Thus UINTA automatically discovers the statistical properties of the signal and can thereby restore a wide spectrum of images and applications. This paper describes the formulation required to minimize the Joint Entropy measure, presents several important practical considerations in estimating image-region statistics, and then presents results on both real and synthetic data.

Jose C Principe - One of the best experts on this subject based on the ideXlab platform.

  • multivariate extension of matrix based renyi s alpha α order Entropy functional
    IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
    Co-Authors: Luis Gonzalo Sanchez Giraldo, Robert Jenssen, Jose C Principe
    Abstract:

    The matrix-based Renyi's $\alpha$ α -order Entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the current theory in the matrix-based Renyi's $\alpha$ α -order Entropy functional only defines the Entropy of a single variable or mutual information between two random variables. In information theory and machine learning communities, one is also frequently interested in multivariate information quantities, such as the multivariate Joint Entropy and different interactive quantities among multiple variables. In this paper, we first define the matrix-based Renyi's $\alpha$ α -order Joint Entropy among multiple variables. We then show how this definition can ease the estimation of various information quantities that measure the interactions among multiple variables, such as interactive information and total correlation. We finally present an application to feature selection to show how our definition provides a simple yet powerful way to estimate a widely-acknowledged intractable quantity from data. A real example on hyperspectral image (HSI) band selection is also provided.

  • multivariate extension of matrix based renyi s alpha order Entropy functional
    arXiv: Information Theory, 2018
    Co-Authors: Luis Gonzalo Sanchez Giraldo, Robert Jenssen, Jose C Principe
    Abstract:

    The matrix-based Renyi's \alpha-order Entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS). However, the current theory in the matrix-based Renyi's \alpha-order Entropy functional only defines the Entropy of a single variable or mutual information between two random variables. In information theory and machine learning communities, one is also frequently interested in multivariate information quantities, such as the multivariate Joint Entropy and different interactive quantities among multiple variables. In this paper, we first define the matrix-based Renyi's \alpha-order Joint Entropy among multiple variables. We then show how this definition can ease the estimation of various information quantities that measure the interactions among multiple variables, such as interactive information and total correlation. We finally present an application to feature selection to show how our definition provides a simple yet powerful way to estimate a widely-acknowledged intractable quantity from data. A real example on hyperspectral image (HSI) band selection is also provided.

  • insights into Entropy as a measure of multivariate variability
    Entropy, 2016
    Co-Authors: Badong Chen, Jianji Wang, Haiquan Zhao, Jose C Principe
    Abstract:

    Entropy has been widely employed as a measure of variability for problems, such as machine learning and signal processing. In this paper, we provide some new insights into the behaviors of Entropy as a measure of multivariate variability. The relationships between multivariate Entropy (Joint or total marginal) and traditional measures of multivariate variability, such as total dispersion and generalized variance, are investigated. It is shown that for the Jointly Gaussian case, the Joint Entropy (or Entropy power) is equivalent to the generalized variance, while total marginal Entropy is equivalent to the geometric mean of the marginal variances and total marginal Entropy power is equivalent to the total dispersion. The smoothed multivariate Entropy (Joint or total marginal) and the kernel density estimation (KDE)-based Entropy estimator (with finite samples) are also studied, which, under certain conditions, will be approximately equivalent to the total dispersion (or a total dispersion estimator), regardless of the data distribution.

Suyash P Awate - One of the best experts on this subject based on the ideXlab platform.

  • unsupervised information theoretic adaptive image filtering for image restoration
    IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006
    Co-Authors: Suyash P Awate, Ross T Whitaker
    Abstract:

    Image restoration is an important and widely studied problem in computer vision and image processing. Various image filtering strategies have been effective, but invariably make strong assumptions about the properties of the signal and/or degradation. Hence, these methods lack the generality to be easily applied to new applications or diverse image collections. This paper describes a novel unsupervised, information-theoretic, adaptive filter (UINTA) that improves the predictability of pixel intensities from their neighborhoods by decreasing their Joint Entropy. In this way, UINTA automatically discovers the statistical properties of the signal and can thereby restore a wide spectrum of images. The paper describes the formulation to minimize the Joint Entropy measure and presents several important practical considerations in estimating neighborhood statistics. It presents a series of results on both real and synthetic data along with comparisons with state-of-the-art techniques, including novel applications to medical image processing.

  • higher order image statistics for unsupervised information theoretic adaptive image filtering
    Computer Vision and Pattern Recognition, 2005
    Co-Authors: Suyash P Awate, Ross T Whitaker
    Abstract:

    The restoration of images is an important and widely studied problem in computer vision and image processing. Various image filtering strategies have been effective, but invariably make strong assumptions about the properties of the signal and/or degradation. Therefore, these methods typically lack the generality to be easily applied to new applications or diverse image collections. This paper describes a novel unsupervised, information-theoretic, adaptive filter (UINTA) that improves the predictability of pixel intensities from their neighborhoods by decreasing the Joint Entropy between them. Thus UINTA automatically discovers the statistical properties of the signal and can thereby restore a wide spectrum of images and applications. This paper describes the formulation required to minimize the Joint Entropy measure, presents several important practical considerations in estimating image-region statistics, and then presents results on both real and synthetic data.