The Experts below are selected from a list of 47271 Experts worldwide ranked by ideXlab platform

Mark John Francis Gales - One of the best experts on this subject based on the ideXlab platform.

  • Minimum phone error training of Precision Matrix models
    IEEE Transactions on Audio Speech and Language Processing, 2006
    Co-Authors: K. C. Sim, Mark John Francis Gales
    Abstract:

    Gaussian mixture models (GMMs) are commonly used as the output density function for large-vocabulary continuous speech recognition (LVCSR) systems. A standard problem when using multivariate GMMs to classify data is how to accurately represent the correlations in the feature vector. Full covariance matrices yield a good model, but dramatically increase the number of model parameters. Hence, diagonal covariance matrices are commonly used. Structured Precision Matrix approximations provide an alternative, flexible, and compact representation. Schemes in this category include the extended maximum likelihood linear transform and subspace for Precision and mean models. This paper examines how these Precision Matrix models can be discriminatively trained and used on state-of-the-art speech recognition tasks. In particular, the use of the minimum phone error criterion is investigated. Implementation issues associated with building LVCSR systems are also addressed. These models are evaluated and compared using large vocabulary continuous telephone speech and broadcast news English tasks.

  • Adaptation of Precision Matrix models on large vocabulary continuous speech recognition
    ICASSP IEEE International Conference on Acoustics Speech and Signal Processing - Proceedings, 2005
    Co-Authors: K. C. Sim, Mark John Francis Gales
    Abstract:

    Recently, structured Precision Matrix models were found to outper- form the conventional diagonal covariance Matrix models. Min- imum phone error discriminative training of these models gave very good unadapted performance on large vocabulary continu- ous speech recognition systems. To obtain state-of-the-art perfor- mance, it is important to apply adaptation techniques efficiently to these models. In this paper, simple row-by-row iterative formulae are described for both MLLR mean and constrained MLLR trans- form estimations of these models. These update formulae are de- rivedwithin the standard expectationmaximisation framework and are guaranteed to increase the likelihood of the adaptation data. Efficient approximate schemes for these adaptation methods are also investigated to further reduce the computation. Experimental results are presented based on the MPE trained Subspace for Pre- cision and Mean models, evaluated on both broadcast news and conversational telephone speech English tasks.

  • ICASSP (1) - Adaptation of Precision Matrix models on large vocabulary continuous speech recognition
    Proceedings. (ICASSP '05). IEEE International Conference on Acoustics Speech and Signal Processing 2005., 1
    Co-Authors: K. C. Sim, Mark John Francis Gales
    Abstract:

    Recently, structured Precision Matrix models were found to outperform the conventional diagonal covariance Matrix models. Minimum phone error discriminative training of these models gave very good unadapted performance on large vocabulary continuous speech recognition systems. To obtain state-of-the-art performance, it is important to apply adaptation techniques efficiently to these models. In this paper, simple row-by-row iterative formulae are described for both MLLR mean and constrained MLLR transform estimations of these models. These update formulae are derived within the standard expectation maximisation framework and are guaranteed to increase the likelihood of the adaptation data. Efficient approximate schemes for these adaptation methods are also investigated to further reduce the computation. Experimental results are presented based on the MPE trained subspace for Precision and mean models, evaluated on both broadcast news and conversational telephone speech English tasks.

K. C. Sim - One of the best experts on this subject based on the ideXlab platform.

  • Minimum phone error training of Precision Matrix models
    IEEE Transactions on Audio Speech and Language Processing, 2006
    Co-Authors: K. C. Sim, Mark John Francis Gales
    Abstract:

    Gaussian mixture models (GMMs) are commonly used as the output density function for large-vocabulary continuous speech recognition (LVCSR) systems. A standard problem when using multivariate GMMs to classify data is how to accurately represent the correlations in the feature vector. Full covariance matrices yield a good model, but dramatically increase the number of model parameters. Hence, diagonal covariance matrices are commonly used. Structured Precision Matrix approximations provide an alternative, flexible, and compact representation. Schemes in this category include the extended maximum likelihood linear transform and subspace for Precision and mean models. This paper examines how these Precision Matrix models can be discriminatively trained and used on state-of-the-art speech recognition tasks. In particular, the use of the minimum phone error criterion is investigated. Implementation issues associated with building LVCSR systems are also addressed. These models are evaluated and compared using large vocabulary continuous telephone speech and broadcast news English tasks.

  • Adaptation of Precision Matrix models on large vocabulary continuous speech recognition
    ICASSP IEEE International Conference on Acoustics Speech and Signal Processing - Proceedings, 2005
    Co-Authors: K. C. Sim, Mark John Francis Gales
    Abstract:

    Recently, structured Precision Matrix models were found to outper- form the conventional diagonal covariance Matrix models. Min- imum phone error discriminative training of these models gave very good unadapted performance on large vocabulary continu- ous speech recognition systems. To obtain state-of-the-art perfor- mance, it is important to apply adaptation techniques efficiently to these models. In this paper, simple row-by-row iterative formulae are described for both MLLR mean and constrained MLLR trans- form estimations of these models. These update formulae are de- rivedwithin the standard expectationmaximisation framework and are guaranteed to increase the likelihood of the adaptation data. Efficient approximate schemes for these adaptation methods are also investigated to further reduce the computation. Experimental results are presented based on the MPE trained Subspace for Pre- cision and Mean models, evaluated on both broadcast news and conversational telephone speech English tasks.

  • ICASSP (1) - Adaptation of Precision Matrix models on large vocabulary continuous speech recognition
    Proceedings. (ICASSP '05). IEEE International Conference on Acoustics Speech and Signal Processing 2005., 1
    Co-Authors: K. C. Sim, Mark John Francis Gales
    Abstract:

    Recently, structured Precision Matrix models were found to outperform the conventional diagonal covariance Matrix models. Minimum phone error discriminative training of these models gave very good unadapted performance on large vocabulary continuous speech recognition systems. To obtain state-of-the-art performance, it is important to apply adaptation techniques efficiently to these models. In this paper, simple row-by-row iterative formulae are described for both MLLR mean and constrained MLLR transform estimations of these models. These update formulae are derived within the standard expectation maximisation framework and are guaranteed to increase the likelihood of the adaptation data. Efficient approximate schemes for these adaptation methods are also investigated to further reduce the computation. Experimental results are presented based on the MPE trained subspace for Precision and mean models, evaluated on both broadcast news and conversational telephone speech English tasks.

  • ICASSP (1) - Basis superposition Precision Matrix modelling for large vocabulary continuous speech recognition
    2004 IEEE International Conference on Acoustics Speech and Signal Processing, 1
    Co-Authors: K. C. Sim, Mjf Gales
    Abstract:

    An important aspect of using Gaussian mixture models in a HMM-based speech recognition systems is the form of the covariance Matrix. One successful approach has been to model the inverse covariance, Precision, Matrix by superimposing multiple bases. This paper presents a general framework of basis superposition. Models are described in terms of parameter tying of the basis coefficients and restrictions in the number of basis. Two forms of parameter tying are described which provide a compact model structure. The first constrains the basis coefficients over multiple basis vectors (or matrices). This is related to the Subspace for Precision and Mean (SPAM) model. The second constrains the basis coefficients over multiple components, yielding as one example heteroscedastic LDA (HLDA). Both maximum likelihood and minimum phone error training of these models are discussed. The performance of various configurations is examined on a conversational telephone speech task, SwitchBoard.

Bo Tang - One of the best experts on this subject based on the ideXlab platform.

  • Non-linear shrinkage-based Precision Matrix estimation for space–time adaptive processing
    Iet Radar Sonar and Navigation, 2017
    Co-Authors: Dandan Zhang, Jun Tang, Bo Tang
    Abstract:

    Space–time adaptive processing (STAP) is usually adopted in airborne and spaceborne radars for clutter suppression and slow moving target detection. However, for traditional sample covariance Matrix inversion algorithm, there are usually not enough independent and identically distributed secondary samples to achieve a satisfactory performance. The diagonal loading methods are usually utilised to solve this finite sample problem. Nevertheless, the performance of diagonal loading STAP can be further improved by directly estimating the Precision Matrix and non-linearly shrinking the eigenvalues of the samples covariance Matrix. This study proposes a non-linear shrinkage-based Precision Matrix estimation algorithm for STAP. Also, a noise power estimation method is provided so that the proposed algorithm still works well even if the noise power is not known previously. The proposed algorithm has much better performance than the diagonal loading methods whether the sample size is smaller or larger than the dimension of the covariance Matrix. Besides, in the condition that the clutter-to-noise ratio is large, which is often encountered in practice, the proposed algorithm also shows a comparatively good performance. Simulations are performed to validate the proposed algorithm.

  • non linear shrinkage based Precision Matrix estimation for space time adaptive processing
    Iet Radar Sonar and Navigation, 2017
    Co-Authors: Dandan Zhang, Jun Tang, Bo Tang
    Abstract:

    Space–time adaptive processing (STAP) is usually adopted in airborne and spaceborne radars for clutter suppression and slow moving target detection. However, for traditional sample covariance Matrix inversion algorithm, there are usually not enough independent and identically distributed secondary samples to achieve a satisfactory performance. The diagonal loading methods are usually utilised to solve this finite sample problem. Nevertheless, the performance of diagonal loading STAP can be further improved by directly estimating the Precision Matrix and non-linearly shrinking the eigenvalues of the samples covariance Matrix. This study proposes a non-linear shrinkage-based Precision Matrix estimation algorithm for STAP. Also, a noise power estimation method is provided so that the proposed algorithm still works well even if the noise power is not known previously. The proposed algorithm has much better performance than the diagonal loading methods whether the sample size is smaller or larger than the dimension of the covariance Matrix. Besides, in the condition that the clutter-to-noise ratio is large, which is often encountered in practice, the proposed algorithm also shows a comparatively good performance. Simulations are performed to validate the proposed algorithm.

Vahe Avagyan - One of the best experts on this subject based on the ideXlab platform.

  • Precision Matrix estimation under data contamination with an application to minimum variance portfolio selection
    Communications in Statistics - Simulation and Computation, 2019
    Co-Authors: Vahe Avagyan, Xiaoling Mei
    Abstract:

    In this article, we consider the problem of estimating the Precision Matrix when the sample data contains cellwise contamination. For the widely employed methodologies (e.g. Graphical Lasso), using...

  • D-Trace estimation of a Precision Matrix with eigenvalue control
    Communications in Statistics - Simulation and Computation, 2019
    Co-Authors: Vahe Avagyan
    Abstract:

    The estimation of a Precision Matrix has an important role in several research fields. In high dimensional settings, one of the most prominent approaches to estimate the Precision Matrix is the Lasso norm penalized convex optimization. This framework guarantees the sparsity of the estimated Precision Matrix. However, it does not control the eigenspectrum of the obtained estimator. Moreover, Lasso penalization shrinks the largest eigenvalues of the estimated Precision Matrix. In this article, we focus on D-trace estimation methodology of a Precision Matrix. We propose imposing a negative trace penalization on the objective function of the D-trace approach, aimed to control the eigenvalues of the estimated Precision Matrix. Through extensive numerical analysis, using simulated and real datasets, we show the advantageous performance of our proposed methodology.

  • Improving the graphical lasso estimation for the Precision Matrix through roots ot the sample convariance Matrix
    Journal of Computational and Graphical Statistics, 2017
    Co-Authors: Vahe Avagyan, Andrés M. Alonso, Francisco J. Nogales
    Abstract:

    In this paper, we focus on the estimation of a high-dimensional Precision Matrix. We propose a simple improvement of the graphical lasso framework (glasso) that is able to attain better statistical performance without sacrificing too much the computational cost. The proposed improvement is based on computing a root of the covariance Matrix to reduce the spread of the associated eigenvalues, and maintains the original convergence rate. Through extensive numerical results, using both simulated and real datasets, we show the proposed modification outperforms the glasso procedure. Finally, our results show that the square-root improvement may be a reasonable choice in practice

  • D-trace estimation of a Precision Matrix using adaptive Lasso penalties
    Advances in Data Analysis and Classification, 2016
    Co-Authors: Vahe Avagyan, Andrés M. Alonso, Francisco J. Nogales
    Abstract:

    The accurate estimation of a Precision Matrix plays a crucial role in the current age of high-dimensional data explosion. To deal with this problem, one of the prominent and commonly used techniques is the \(\ell _1\) norm (Lasso) penalization for a given loss function. This approach guarantees the sparsity of the Precision Matrix estimate for properly selected penalty parameters. However, the \(\ell _1\) norm penalization often fails to control the bias of obtained estimator because of its overestimation behavior. In this paper, we introduce two adaptive extensions of the recently proposed \(\ell _1\) norm penalized D-trace loss minimization method. They aim at reducing the produced bias in the estimator. Extensive numerical results, using both simulated and real datasets, show the advantage of our proposed estimators.

  • D-Trace Precision Matrix estimator with eigenvalue control
    2016
    Co-Authors: Vahe Avagyan
    Abstract:

    The estimation of a Precision Matrix has an important role in several research fields. In high-dimensional settings, one of the most prominent approaches to estimate the Precision Matrix is the ɭ₁ (Lasso) norm penalized convex optimization. This framework guarantees the sparsity of the estimated Precision Matrix. However, it does not control the eigenspectrum of the obtained estimator, and, moreover, it shrinks the largest eigenvalues of the estimated Precision Matrix. In this paper, we focus on D-trace Precision Matrix methodology. We propose imposing a negative trace penalization on the objective function of the D-trace approach, aimed to control the eigenvalues. Through extensive numerical analysis, using simulated and real datasets, we show the advantageous performance of our proposed methodology.

Tommaso Cai - One of the best experts on this subject based on the ideXlab platform.

  • Estimating Sparse Precision Matrix: Optimal Rates of Convergence and Adaptive Estimation
    The Annals of Statistics, 2016
    Co-Authors: Tommaso Cai, Weidong Liu, Harrison H. Zhou
    Abstract:

    Precision Matrix is of significant importance in a wide range of applications in multivariate analysis. This paper considers adaptive minimax estimation of sparse Precision matrices in the high dimensional setting. Optimal rates of convergence are established for a range of Matrix norm losses. A fully data driven estimator based on adaptive constrained l1 minimization is proposed and its rate of convergence is obtained over a collection of parameter spaces. The estimator, called ACLIME, is easy to implement and performs well numerically. A major step in establishing the minimax rate of convergence is the derivation of a rate-sharp lower bound. A “two-directional” lower bound technique is applied to obtain the minimax lower bound. The upper and lower bounds together yield the optimal rates of convergence for sparse Precision Matrix estimation and show that the ACLIME estimator is adaptively minimax rate optimal for a collection of parameter spaces and a range of Matrix norm losses simultaneously.

  • Estimating Sparse Precision Matrix: Optimal Rates of Convergence and Adaptive Estimation
    arXiv: Statistics Theory, 2012
    Co-Authors: Tommaso Cai, Weidong Liu, Harrison H. Zhou
    Abstract:

    Precision Matrix is of significant importance in a wide range of applications in multivariate analysis. This paper considers adaptive minimax estimation of sparse Precision matrices in the high dimensional setting. Optimal rates of convergence are established for a range of Matrix norm losses. A fully data driven estimator based on adaptive constrained $\ell_1$ minimization is proposed and its rate of convergence is obtained over a collection of parameter spaces. The estimator, called ACLIME, is easy to implement and performs well numerically. A major step in establishing the minimax rate of convergence is the derivation of a rate-sharp lower bound. A "two-directional" lower bound technique is applied to obtain the minimax lower bound. The upper and lower bounds together yield the optimal rates ofconvergence for sparse Precision Matrix estimation and show that the ACLIME estimator is adaptively minimax rate optimal for a collection of parameter spaces and a range of Matrix norm losses simultaneously.

  • Covariate-Adjusted Precision Matrix Estimation with an Application in Genetical Genomics.
    Biometrika, 2012
    Co-Authors: Tommaso Cai, Weidong Liu, Jichun Xie
    Abstract:

    Motivated by analysis of genetical genomics data, we introduce a sparse high-dimensional multivariate regression model for studying conditional independence relationships among a set of genes adjusting for possible genetic effects. The Precision Matrix in the model specifies a covariate-adjusted Gaussian graph, which presents the conditional dependence structure of gene expression after the confounding genetic effects on gene expression are taken into account. We present a covariate-adjusted Precision Matrix estimation method using a constrained e 1 minimization, which can be easily implemented by linear programming. Asymptotic convergence rates in various Matrix norms and sign consistency are established for the estimators of the regression coefficients and the Precision Matrix, allowing both the number of genes and the number of the genetic variants to diverge. Simulation shows that the proposed method results in significant improvements in both Precision Matrix estimation and graphical structure selection when compared to the standard Gaussian graphical model assuming constant means. The proposed method is applied to yeast genetical genomics data for the identification of the gene network among a set of genes in the mitogen-activated protein kinase pathway. Copyright 2013, Oxford University Press.

  • a constrained l1 minimization approach to sparse Precision Matrix estimation
    Journal of the American Statistical Association, 2011
    Co-Authors: Tommaso Cai, Weidong Liu, Xi Luo
    Abstract:

    This article proposes a constrained l1 minimization method for estimating a sparse inverse covariance Matrix based on a sample of n iid p-variate random variables. The resulting estimator is shown to have a number of desirable properties. In particular, the rate of convergence between the estimator and the true s-sparse Precision Matrix under the spectral norm is when the population distribution has either exponential-type tails or polynomial-type tails. We present convergence rates under the elementwise l∞ norm and Frobenius norm. In addition, we consider graphical model selection. The procedure is easily implemented by linear programming. Numerical performance of the estimator is investigated using both simulated and real data. In particular, the procedure is applied to analyze a breast cancer dataset and is found to perform favorably compared with existing methods.

  • a constrained l1 minimization approach to sparse Precision Matrix estimation
    arXiv: Methodology, 2011
    Co-Authors: Tommaso Cai, Weidong Liu, Xi Luo
    Abstract:

    A constrained L1 minimization method is proposed for estimating a sparse inverse covariance Matrix based on a sample of $n$ iid $p$-variate random variables. The resulting estimator is shown to enjoy a number of desirable properties. In particular, it is shown that the rate of convergence between the estimator and the true $s$-sparse Precision Matrix under the spectral norm is $s\sqrt{\log p/n}$ when the population distribution has either exponential-type tails or polynomial-type tails. Convergence rates under the elementwise $L_{\infty}$ norm and Frobenius norm are also presented. In addition, graphical model selection is considered. The procedure is easily implementable by linear programming. Numerical performance of the estimator is investigated using both simulated and real data. In particular, the procedure is applied to analyze a breast cancer dataset. The procedure performs favorably in comparison to existing methods.