The Experts below are selected from a list of 360 Experts worldwide ranked by ideXlab platform

Warren B. Powell - One of the best experts on this subject based on the ideXlab platform.

  • Dirichlet Process Mixtures of Generalized Linear Models
    Journal of Machine Learning Research, 2011
    Co-Authors: Lauren A. Hannah, David M. Blei, Warren B. Powell
    Abstract:

    We propose Dirichlet Process mixtures of Generalized Linear Models (DP-GLM), a new class of methods for nonparametric regression. Given a data set of input-response pairs, the DP-GLM produces a global model of the joint distribution through a mixture of local Generalized Linear Models. DP-GLMs allow both continuous and categorical inputs, and can model the same class of responses that can be modeled with a Generalized Linear model. We study the properties of the DP-GLM, and show why it provides better predictions and density estimates than existing Dirichlet process mixture regression Models. We give conditions for weak consistency of the joint distribution and pointwise consistency of the regression estimate.

  • dirichlet process mixtures of Generalized Linear Models
    arXiv: Machine Learning, 2009
    Co-Authors: Lauren A. Hannah, David M. Blei, Warren B. Powell
    Abstract:

    We propose Dirichlet Process mixtures of Generalized Linear Models (DP-GLM), a new method of nonparametric regression that accommodates continuous and categorical inputs, and responses that can be modeled by a Generalized Linear model. We prove conditions for the asymptotic unbiasedness of the DP-GLM regression mean function estimate. We also give examples for when those conditions hold, including Models for compactly supported continuous distributions and a model with continuous covariates and categorical response. We empirically analyze the properties of the DP-GLM and why it provides better results than existing Dirichlet process mixture regression Models. We evaluate DP-GLM on several data sets, comparing it to modern methods of nonparametric regression like CART, Bayesian trees and Gaussian processes. Compared to existing techniques, the DP-GLM provides a single model (and corresponding inference algorithms) that performs well in many regression settings.

Lauren A. Hannah - One of the best experts on this subject based on the ideXlab platform.

  • Dirichlet Process Mixtures of Generalized Linear Models
    Journal of Machine Learning Research, 2011
    Co-Authors: Lauren A. Hannah, David M. Blei, Warren B. Powell
    Abstract:

    We propose Dirichlet Process mixtures of Generalized Linear Models (DP-GLM), a new class of methods for nonparametric regression. Given a data set of input-response pairs, the DP-GLM produces a global model of the joint distribution through a mixture of local Generalized Linear Models. DP-GLMs allow both continuous and categorical inputs, and can model the same class of responses that can be modeled with a Generalized Linear model. We study the properties of the DP-GLM, and show why it provides better predictions and density estimates than existing Dirichlet process mixture regression Models. We give conditions for weak consistency of the joint distribution and pointwise consistency of the regression estimate.

  • dirichlet process mixtures of Generalized Linear Models
    arXiv: Machine Learning, 2009
    Co-Authors: Lauren A. Hannah, David M. Blei, Warren B. Powell
    Abstract:

    We propose Dirichlet Process mixtures of Generalized Linear Models (DP-GLM), a new method of nonparametric regression that accommodates continuous and categorical inputs, and responses that can be modeled by a Generalized Linear model. We prove conditions for the asymptotic unbiasedness of the DP-GLM regression mean function estimate. We also give examples for when those conditions hold, including Models for compactly supported continuous distributions and a model with continuous covariates and categorical response. We empirically analyze the properties of the DP-GLM and why it provides better results than existing Dirichlet process mixture regression Models. We evaluate DP-GLM on several data sets, comparing it to modern methods of nonparametric regression like CART, Bayesian trees and Gaussian processes. Compared to existing techniques, the DP-GLM provides a single model (and corresponding inference algorithms) that performs well in many regression settings.

Sara Van De Geer - One of the best experts on this subject based on the ideXlab platform.

  • high dimensional Generalized Linear Models and the lasso
    Annals of Statistics, 2008
    Co-Authors: Sara Van De Geer
    Abstract:

    We consider high-dimensional Generalized Linear Models with Lipschitz loss functions, and prove a nonasymptotic oracle inequality for the empirical risk minimizer with Lasso penalty. The penalty is based on the coefficients in the Linear predictor, after normalization with the empirical norm. The examples include logistic regression, density estimation and classification with hinge loss. Least squares regression is also discussed.

John A. Nelder - One of the best experts on this subject based on the ideXlab platform.

  • Generalized Linear Models with random effects unified analysis via h likelihood
    2006
    Co-Authors: Youngjo Lee, John A. Nelder, Yudi Pawitan
    Abstract:

    LIST OF NOTATIONS PREFACE INTRODUCTION CLASSICAL LIKELIHOOD THEORY Definition Quantities derived from the likelihood Profile likelihood Distribution of the likelihood-ratio statistic Distribution of the MLE and the Wald statistic Model selection Marginal and conditional likelihoods Higher-order approximations Adjusted profile likelihood Bayesian and likelihood methods Jacobian in likelihood methods Generalized Linear Models Linear Models Generalized Linear Models Model checking Examples QUASI-LIKELIHOOD Examples Iterative weighted least squares Asymptotic inference Dispersion Models Extended Quasi-likelihood Joint GLM of mean and dispersion Joint GLMs for quality improvement EXTENDED LIKELIHOOD INFERENCES Two kinds of likelihood Inference about the fixed parameters Inference about the random parameters Optimality in random-parameter estimation Canonical scale, h-likelihood and joint inference Statistical prediction Regression as an extended model Missing or incomplete-data problems Is marginal likelihood enough for inference about fixed parameters? Summary: likelihoods in extended framework NORMAL Linear MIXED Models Developments of normal mixed Linear Models Likelihood estimation of fixed parameters Classical estimation of random effects H-likelihood approach Example Invariance and likelihood inference HIERARCHICAL GLMS HGLMs H-likelihood Inferential procedures using h-likelihood Penalized quasi-likelihood Deviances in HGLMs Examples Choice of random-effect scale HGLMS WITH STRUCTURED DISPERSION HGLMs with structured dispersion Quasi-HGLMs Examples CORRELATED RANDOM EFFECTS FOR HGLMS HGLMs with correlated random effects Random effects described by fixed L matrices Random effects described by a covariance matrix Random effects described by a precision matrix Fitting and model-checking Examples Twin and family data Ascertainment problem SMOOTHING Spline Models Mixed model framework Automatic smoothing Non-Gaussian smoothing RANDOM-EFFECT Models FOR SURVIVAL DATA Proportional-hazard model Frailty Models and the associated h-likelihood *Mixed Linear Models with censoring Extensions Proofs DOUBLE HGLMs DHGLMs Models for finance data H-likelihood procedure for fitting DHGLMs Random effects in the ? component Examples FURTHER TOPICS Model for multivariate responses Joint model for continuous and binary data Joint model for repeated measures and survival time Missing data in longitudinal studies Denoising signals by imputation REFERENCE DATA INDEX AUTHOR INDEX SUBJECT INDEX

  • double hierarchical Generalized Linear Models with discussion
    Journal of The Royal Statistical Society Series C-applied Statistics, 2006
    Co-Authors: Youngjo Lee, John A. Nelder
    Abstract:

    Summary.  We propose a class of double hierarchical Generalized Linear Models in which random effects can be specified for both the mean and dispersion. Heteroscedasticity between clusters can be modelled by introducing random effects in the dispersion model, as is heterogeneity between clusters in the mean model. This class will, among other things, enable Models with heavy-tailed distributions to be explored, providing robust estimation against outliers. The h-likelihood provides a unified framework for this new class of Models and gives a single algorithm for fitting all members of the class. This algorithm does not require quadrature or prior probabilities.

  • Hierarchical Generalized Linear Models
    Journal of the Royal Statistical Society: Series B (Methodological), 1996
    Co-Authors: Youngjo Lee, John A. Nelder
    Abstract:

    We consider hierarchical Generalized Linear Models which allow extra error components in the Linear predictors of Generalized Linear Models. The distribution of these components is not restricted to be normal; this allows a broader class of Models, which includes Generalized Linear mixed Models. We use a generalization of Henderson's joint likelihood, called a hierarchical or h-likelihood, for inferences from hierarchical Generalized Linear Models. This avoids the integration that is necessary when marginal likelihood is used. Under appropriate conditions maximizing the h-likelihood gives fixed effect estimators that are asymptotically equivalent to those obtained from the use of marginal likelihood; at the same time we obtain the random effect estimates that are asymptotically best unbiased predictors. An adjusted profile h-likelihood is shown to give the required generalization of restricted maximum likelihood for the estimation of dispersion components. A scaled deviance test for the goodness of fit, a model selection criterion for choosing between various dispersion Models and a graphical method for checking the distributional assumption of random effects are proposed. The ideas of quasi-likelihood and extended quasi-likelihood are Generalized to the new class. We give examples of the Poisson-gamma, binomial-beta and gamma-inverse gamma hierarchical Generalized Linear Models. A resolution is proposed for the apparent difference between population-averaged and subject-specific Models. A unified framework is provided for viewing and extending many existing methods.

Trevor Hastie - One of the best experts on this subject based on the ideXlab platform.

  • Regularization Paths for Generalized Linear Models via Coordinate Descent.
    Journal of statistical software, 2010
    Co-Authors: Jerome Friedman, Trevor Hastie, Rob Tibshirani
    Abstract:

    We develop fast algorithms for estimation of Generalized Linear Models with convex penalties. The Models include Linear regression, two-class logistic regression, and multinomial regression problems while the penalties include ℓ(1) (the lasso), ℓ(2) (ridge regression) and mixtures of the two (the elastic net). The algorithms use cyclical coordinate descent, computed along a regularization path. The methods can handle large problems and can also deal efficiently with sparse features. In comparative timings we find that the new algorithms are considerably faster than competing methods.

  • l1 regularization path algorithm for Generalized Linear Models
    Journal of The Royal Statistical Society Series B-statistical Methodology, 2007
    Co-Authors: Mee Young Park, Trevor Hastie
    Abstract:

    Summary. We introduce a path following algorithm for L1‐regularized Generalized Linear Models. The L1‐regularization procedure is useful especially because it, in effect, selects variables according to the amount of penalization on the L1‐norm of the coefficients, in a manner that is less greedy than forward selection–backward deletion. The Generalized Linear model path algorithm efficiently computes solutions along the entire regularization path by using the predictor–corrector method of convex optimization. Selecting the step length of the regularization parameter is critical in controlling the overall accuracy of the paths; we suggest intuitive and flexible strategies for choosing appropriate values. We demonstrate the implementation with several simulated and real data sets.

  • reduced rank vector Generalized Linear Models
    Statistical Modelling, 2003
    Co-Authors: Thomas W Yee, Trevor Hastie
    Abstract:

    Reduced-rank regression is a method with great potential for dimension reduction but has found few applications in applied statistics. To address this, reduced-rank regression is proposed for the class of vector Generalized Linear Models (VGLMs), which is very large. The resulting class, which we call reduced-rank VGLMs (RR-VGLMs), enables the benefits of reduced-rank regression to be conveyed to a wide range of data types, including categorical data. RR-VGLMs are illustrated by focussing on Models for categorical data, and especially the multinomial logit model. General algorithmic details are provided and software written by the first author is described. The reduced-rank multinomial logit model is illustrated with real data in two contexts: a regression analysis of workforce data and a classification problem.