The Experts below are selected from a list of 46863 Experts worldwide ranked by ideXlab platform
Daniel Yekutieli - One of the best experts on this subject based on the ideXlab platform.
-
hierarchical False Discovery rate controlling methodology
Journal of the American Statistical Association, 2008Co-Authors: Daniel YekutieliAbstract:We discuss methodology for controlling the False Discovery rate (FDR) in complex large-scale studies that involve testing multiple families of hypotheses; the tested hypotheses are arranged in a tree of disjoint subfamilies, and the subfamilies of hypotheses are hierarchically tested by the Benjamini and Hochberg FDR-controlling (BH) procedure. We derive an approximation for the multiple family FDR for independently distributed test statistics: q, the level at which the BH procedure is applied, times the number of families tested plus the number of discoveries, divided by the number of discoveries plus 1. We provide a universal bound for the FDR of the discoveries in the new hierarchical testing approach, 2 × 1.44 × q, and demonstrate in simulations that when the data has an hierarchical structure the new testing approach can be considerably more powerful than the BH procedure.
-
adaptive linear step up procedures that control the False Discovery rate
Biometrika, 2006Co-Authors: Yoav Benjamini, Abba M Krieger, Daniel YekutieliAbstract:The linear step-up multiple testing procedure controls the False Discovery rate at the desired level q for independent and positively dependent test statistics. When all null hypotheses are true, and the test statistics are independent and continuous, the bound is sharp. When some of the null hypotheses are not true, the procedure is conservative by a factor which is the proportion m-sub-0/m of the true null hypotheses among the hypotheses. We provide a new two-stage procedure in which the linear step-up procedure is used in stage one to estimate m-sub-0, providing a new level q′ which is used in the linear step-up procedure in the second stage. We prove that a general form of the two-stage procedure controls the False Discovery rate at the desired level q. This framework enables us to study analytically the properties of other procedures that exist in the literature. A simulation study is presented that shows that two-stage adaptive procedures improve in power over the original procedure, mainly because they provide tighter control of the False Discovery rate. We further study the performance of the current suggestions, some variations of the procedures, and previous suggestions, in the case where the test statistics are positively dependent, a case for which the original procedure controls the False Discovery rate. In the setting studied here the newly proposed two-stage procedure is the only one that controls the False Discovery rate. The procedures are illustrated with two examples of biological importance. Copyright 2006, Oxford University Press.
-
adaptive linear step up procedures that control the False Discovery rate
Biometrika, 2006Co-Authors: Yoav Benjamini, Abba M Krieger, Daniel YekutieliAbstract:We provide a new two-stage procedure in which the linear step-up procedure is used in stage one to estimate mo, providing a new level q' which is used in the linear step-up procedure in the second stage. We prove that a general form of the two-stage procedure controls the False Discovery rate at the desired level q. This framework enables us to study analytically the properties of other procedures that exist in the literature. A simulation study is presented that shows that two-stage adaptive procedures improve in power over the original procedure, mainly because they provide tighter control of the False Discovery rate. We further study the performance of the current suggestions, some variations of the procedures, and previous suggestions, in the case where the test statistics are positively dependent, a case for which the original procedure controls the False Discovery rate. In the setting studied here the newly proposed two-stage procedure is the only one that controls the False Discovery rate. The procedures are illustrated with two examples of biological importance.
-
quantitative trait loci analysis using the False Discovery rate
Genetics, 2005Co-Authors: Yoav Benjamini, Daniel YekutieliAbstract:False Discovery rate control has become an essential tool in any study that has a very large multiplicity problem. False Discovery rate-controlling procedures have also been found to be very effective in QTL analysis, ensuring reproducible results with few Falsely discovered linkages and offering increased power to discover QTL, although their acceptance has been slower than in microarray analysis, for example. The reason is partly because the methodological aspects of applying the False Discovery rate to QTL mapping are not well developed. Our aim in this work is to lay a solid foundation for the use of the False Discovery rate in QTL mapping. We review the False Discovery rate criterion, the appropriate interpretation of the FDR, and alternative formulations of the FDR that appeared in the statistical and genetics literature. We discuss important features of the FDR approach, some stemming from new developments in FDR theory and methodology, which deem it especially useful in linkage analysis. We review False Discovery rate-controlling procedures—the BH, the resampling procedure, and the adaptive two-stage procedure—and discuss the validity of these procedures in single- and multiple-trait QTL mapping. Finally we argue that the control of the False Discovery rate has an important role in suggesting, indicating the significance of, and confirming QTL and present guidelines for its use.
-
False Discovery rate adjusted multiple confidence intervals for selected parameters
Journal of the American Statistical Association, 2005Co-Authors: Yoav Benjamini, Daniel YekutieliAbstract:Often in applied research, confidence intervals (CIs) are constructed or reported only for parameters selected after viewing the data. We show that such selected intervals fail to provide the assumed coverage probability. By generalizing the False Discovery rate (FDR) approach from multiple testing to selected multiple CIs, we suggest the False coverage-statement rate (FCR) as a measure of interval coverage following selection. A general procedure is then introduced, offering FCR control at level q under any selection rule. The procedure constructs a marginal CI for each selected parameter, but instead of the confidence level 1 − q being used marginally, q is divided by the number of parameters considered and multiplied by the number selected. If we further use the FDR controlling testing procedure of Benjamini and Hochberg for selecting the parameters, the newly suggested procedure offers CIs that are dual to the testing procedure and are shown to be optimal in the independent case. Under the positive re...
Yoav Benjamini - One of the best experts on this subject based on the ideXlab platform.
-
weighted False Discovery rate controlling procedures for clinical trials
Biostatistics, 2017Co-Authors: Yoav Benjamini, Rami CohenAbstract:Having identified that the lack of replicability of results in earlier phases of clinical medical research stems largely from unattended selective inference, we offer a new hierarchical weighted False Discovery rate controlling testing procedure alongside the single-level weighted procedure. These address the special structure of clinical research, where the comparisons of treatments involve both primary and secondary endpoints, by assigning weights that reflect the relative importance of the endpoints in the error being controlled. In the hierarchical method, the primary endpoints and a properly weighted intersection hypothesis that represents all secondary endpoints are tested. Should the intersection hypothesis be among the rejected, individual secondary endpoints are tested. We identify configurations where each of the two procedures has the advantage. Both offer higher power than competing hierarchical (gatekeeper) familywise error-rate controlling procedures being used for drug approval. By their design, the advantage of the proposed methods is the increased power to discover effects on secondary endpoints, without giving up the rigor of addressing their multiplicity.
-
discovering the False Discovery rate
Journal of The Royal Statistical Society Series B-statistical Methodology, 2010Co-Authors: Yoav BenjaminiAbstract:I describe the background for the paper 'Controlling the False Discovery rate: a new and powerful approach to multiple comparisons' by Benjamini and Hochberg that was published in the "Journal of the Royal Statistical Society", Series B, in 1995. I review the progress since made on the False Discovery rate, as well as the major conceptual developments that followed. Copyright (c) 2010 Royal Statistical Society.
-
adaptive linear step up procedures that control the False Discovery rate
Biometrika, 2006Co-Authors: Yoav Benjamini, Abba M Krieger, Daniel YekutieliAbstract:The linear step-up multiple testing procedure controls the False Discovery rate at the desired level q for independent and positively dependent test statistics. When all null hypotheses are true, and the test statistics are independent and continuous, the bound is sharp. When some of the null hypotheses are not true, the procedure is conservative by a factor which is the proportion m-sub-0/m of the true null hypotheses among the hypotheses. We provide a new two-stage procedure in which the linear step-up procedure is used in stage one to estimate m-sub-0, providing a new level q′ which is used in the linear step-up procedure in the second stage. We prove that a general form of the two-stage procedure controls the False Discovery rate at the desired level q. This framework enables us to study analytically the properties of other procedures that exist in the literature. A simulation study is presented that shows that two-stage adaptive procedures improve in power over the original procedure, mainly because they provide tighter control of the False Discovery rate. We further study the performance of the current suggestions, some variations of the procedures, and previous suggestions, in the case where the test statistics are positively dependent, a case for which the original procedure controls the False Discovery rate. In the setting studied here the newly proposed two-stage procedure is the only one that controls the False Discovery rate. The procedures are illustrated with two examples of biological importance. Copyright 2006, Oxford University Press.
-
adaptive linear step up procedures that control the False Discovery rate
Biometrika, 2006Co-Authors: Yoav Benjamini, Abba M Krieger, Daniel YekutieliAbstract:We provide a new two-stage procedure in which the linear step-up procedure is used in stage one to estimate mo, providing a new level q' which is used in the linear step-up procedure in the second stage. We prove that a general form of the two-stage procedure controls the False Discovery rate at the desired level q. This framework enables us to study analytically the properties of other procedures that exist in the literature. A simulation study is presented that shows that two-stage adaptive procedures improve in power over the original procedure, mainly because they provide tighter control of the False Discovery rate. We further study the performance of the current suggestions, some variations of the procedures, and previous suggestions, in the case where the test statistics are positively dependent, a case for which the original procedure controls the False Discovery rate. In the setting studied here the newly proposed two-stage procedure is the only one that controls the False Discovery rate. The procedures are illustrated with two examples of biological importance.
-
adapting to unknown sparsity by controlling the False Discovery rate
Annals of Statistics, 2006Co-Authors: Felix Abramovich, Yoav Benjamini, David L Donoho, Iain M. JohnstoneAbstract:We attempt to recover an n-dimensional vector observed in white noise, where n is large and the vector is known to be sparse, but the degree of sparsity is unknown. We consider three different ways of defining sparsity of a vector: using the fraction of nonzero terms; imposing power-law decay bounds on the ordered entries; and controlling the lp norm for p small. We obtain a procedure which is asymptotically minimax for l r loss, simultaneously throughout a range of such sparsity classes. The optimal procedure is a data-adaptive thresholding scheme, driven by control of the False Discovery Rate (FDR). FDR control is a relatively recent innovation in simultaneous testing, ensuring that at most a certain fraction of the rejected null hypotheses will correspond to False rejections. In our treatment, the FDR control parameter qn also plays a determining role in asymptotic minimaxity. If q = lim qn ∈ [0,1/2] and also qn > γ/log(n) we get sharp asymptotic minimaxity, simultaneously, over a wide range of sparse parameter spaces and loss functions. On the other hand, q = lim qn ∈ (1/2,1], forces the risk to exceed the minimax risk by a factor growing with q. To our knowledge, this relation between ideas in simultaneous inference and asymptotic decision theory is new. Our work provides a new perspective on a class of model selection rules which has been introduced recently by several authors. These new rules impose complexity penalization of the form 2 � log( potential model size / actual model size ). We exhibit a close connection with FDR-controlling procedures under stringent control of the False Discovery rate.
James G Scott - One of the best experts on this subject based on the ideXlab platform.
-
False Discovery rate smoothing
Journal of the American Statistical Association, 2018Co-Authors: Wesley Tansey, Oluwasanmi Koyejo, Russell Alan Poldrack, James G ScottAbstract:ABSTRACTWe present False Discovery rate (FDR) smoothing, an empirical-Bayes method for exploiting spatial structure in large multiple-testing problems. FDR smoothing automatically finds spatially localized regions of significant test statistics. It then relaxes the threshold of statistical significance within these regions, and tightens it elsewhere, in a manner that controls the overall False Discovery rate at a given level. This results in increased power and cleaner spatial separation of signals from noise. The approach requires solving a nonstandard high-dimensional optimization problem, for which an efficient augmented-Lagrangian algorithm is presented. In simulation studies, FDR smoothing exhibits state-of-the-art performance at modest computational cost. In particular, it is shown to be far more robust than existing methods for spatially dependent multiple testing. We also apply the method to a dataset from an fMRI experiment on spatial working memory, where it detects patterns that are much more b...
-
False Discovery rate regression an application to neural synchrony detection in primary visual cortex
Journal of the American Statistical Association, 2015Co-Authors: James G Scott, Ryan C Kelly, Matthew A. Smith, Pengcheng Zhou, Robert E. KassAbstract:This article introduces False Discovery rate regression, a method for incorporating covariate information into large-scale multiple-testing problems. FDR regression estimates a relationship between test-level covariates and the prior probability that a given observation is a signal. It then uses this estimated relationship to inform the outcome of each test in a way that controls the overall False Discovery rate at a prespecified level. This poses many subtle issues at the interface between inference and computation, and we investigate several variations of the overall approach. Simulation evidence suggests that: (1) when covariate effects are present, FDR regression improves power for a fixed False-Discovery rate; and (2) when covariate effects are absent, the method is robust, in the sense that it does not lead to inflated error rates. We apply the method to neural recordings from primary visual cortex. The goal is to detect pairs of neurons that exhibit fine-time-scale interactions, in the sense that they fire together more often than expected due to chance. Our method detects roughly 50% more synchronous pairs versus a standard FDR-controlling analysis. The companion R package FDRreg implements all methods described in the article. Supplementary materials for this article are available online.
-
False Discovery rate regression an application to neural synchrony detection in primary visual cortex
Journal of the American Statistical Association, 2015Co-Authors: James G Scott, Ryan C Kelly, Matthew A. Smith, Pengcheng Zhou, Robert E. KassAbstract:This article introduces False Discovery rate regression, a method for incorporating covariate information into large-scale multiple-testing problems. FDR regression estimates a relationship between test-level covariates and the prior probability that a given observation is a signal. It then uses this estimated relationship to inform the outcome of each test in a way that controls the overall False Discovery rate at a prespecified level. This poses many subtle issues at the interface between inference and computation, and we investigate several variations of the overall approach. Simulation evidence suggests that: (1) when covariate effects are present, FDR regression improves power for a fixed False-Discovery rate; and (2) when covariate effects are absent, the method is robust, in the sense that it does not lead to inflated error rates. We apply the method to neural recordings from primary visual cortex. The goal is to detect pairs of neurons that exhibit fine-time-scale interactions, in the sense that t...
-
False Discovery rate smoothing
arXiv: Methodology, 2014Co-Authors: Wesley Tansey, Oluwasanmi Koyejo, Russell Alan Poldrack, James G ScottAbstract:We present False Discovery rate smoothing, an empirical-Bayes method for exploiting spatial structure in large multiple-testing problems. FDR smoothing automatically finds spatially localized regions of significant test statistics. It then relaxes the threshold of statistical significance within these regions, and tightens it elsewhere, in a manner that controls the overall False-Discovery rate at a given level. This results in increased power and cleaner spatial separation of signals from noise. The approach requires solving a non-standard high-dimensional optimization problem, for which an efficient augmented-Lagrangian algorithm is presented. In simulation studies, FDR smoothing exhibits state-of-the-art performance at modest computational cost. In particular, it is shown to be far more robust than existing methods for spatially dependent multiple testing. We also apply the method to a data set from an fMRI experiment on spatial working memory, where it detects patterns that are much more biologically plausible than those detected by standard FDR-controlling methods. All code for FDR smoothing is publicly available in Python and R.
-
False Discovery rate regression an application to neural synchrony detection in primary visual cortex
arXiv: Methodology, 2013Co-Authors: James G Scott, Ryan C Kelly, Matthew A. Smith, Pengcheng Zhou, Robert E. KassAbstract:Many approaches for multiple testing begin with the assumption that all tests in a given study should be combined into a global False-Discovery-rate analysis. But this may be inappropriate for many of today's large-scale screening problems, where auxiliary information about each test is often available, and where a combined analysis can lead to poorly calibrated error rates within different subsets of the experiment. To address this issue, we introduce an approach called False-Discovery-rate regression that directly uses this auxiliary information to inform the outcome of each test. The method can be motivated by a two-groups model in which covariates are allowed to influence the local False Discovery rate, or equivalently, the posterior probability that a given observation is a signal. This poses many subtle issues at the interface between inference and computation, and we investigate several variations of the overall approach. Simulation evidence suggests that: (1) when covariate effects are present, FDR regression improves power for a fixed False-Discovery rate; and (2) when covariate effects are absent, the method is robust, in the sense that it does not lead to inflated error rates. We apply the method to neural recordings from primary visual cortex. The goal is to detect pairs of neurons that exhibit fine-time-scale interactions, in the sense that they fire together more often than expected due to chance. Our method detects roughly 50% more synchronous pairs versus a standard FDR-controlling analysis. The companion R package FDRreg implements all methods described in the paper.
Larry Wasserman - One of the best experts on this subject based on the ideXlab platform.
-
Outlier Detection and False Discovery Rates for Whole-genome DNA Matching
2018Co-Authors: Jung-ying Tzeng, W. Byerley, B. Devlin, Kathryn Roeder, Larry WassermanAbstract:We define a statistic, called the matching statistic, for locating regions of the genome that exhibit excess similarity among cases when compared to controls. Such regions are reasonable candidates for harboring disease genes. We find the asymptotic distribution of the statistic while accounting for correlations among sampled individuals. We then use the Benjamini and Hochberg False Discovery rate (FDR) method for multiple hypothesis testing to find regions of excess sharing. The p-values for each region involve estimated nuisance parameters. Under appropriate conditions, we show that the FDR method based on p-values and with estimated nuisance parameters asymptotically preserves the FDR property. Finally, we apply the method to a pilot study on schizophrenia.
-
exceedance control of the False Discovery proportion
Journal of the American Statistical Association, 2006Co-Authors: Christopher R. Genovese, Larry WassermanAbstract:Multiple testing methods to control the False Discovery rate, the expected proportion of Falsely rejected null hypotheses among all rejections, have received much attention. It can be valuable to control not the mean of this False Discovery proportion (FDP), but rather the probability that the FDP exceeds a specified bound. In this article we construct a general class of methods for exceedance control of FDP based on inverting tests of uniformity. The method also produces a confidence envelope for the FDP as a function of rejection threshold. We discuss how to select a procedure with good power.
-
operating characteristics and extensions of the False Discovery rate procedure
Journal of The Royal Statistical Society Series B-statistical Methodology, 2002Co-Authors: Christopher R. Genovese, Larry WassermanAbstract:We investigate the operating characteristics of the Benjamini-Hochberg False Discovery rate procedure for multiple testing. This is a distribution-free method that controls the expected fraction of Falsely rejected null hypotheses among those rejected. The paper provides a framework for understanding more about this procedure. We first study the asymptotic properties of the `deciding point' "D" that determines the critical "p"-value. From this, we obtain explicit asymptotic expressions for a particular risk function. We introduce the dual notion of False non-rejections and we consider a risk function that combines the False Discovery rate and False non-rejections. We also consider the optimal procedure with respect to a measure of conditional risk. Copyright 2002 Royal Statistical Society.
-
operating characteristics and extensions of the False Discovery rate procedure
Journal of The Royal Statistical Society Series B-statistical Methodology, 2002Co-Authors: Christopher R. Genovese, Larry WassermanAbstract:Summary. We investigate the operating characteristics of the Benjamini–Hochberg False Discovery rate procedure for multiple testing. This is a distribution-free method that controls the expected fraction of Falsely rejected null hypotheses among those rejected. The paper provides a framework for understanding more about this procedure. We first study the asymptotic properties of the ‘deciding point’ D that determines the critical p-value. From this, we obtain explicit asymptotic expressions for a particular risk function. We introduce the dual notion of False non-rejections and we consider a risk function that combines the False Discovery rate and False non-rejections. We also consider the optimal procedure with respect to a measure of conditional risk.
Christopher R. Genovese - One of the best experts on this subject based on the ideXlab platform.
-
exceedance control of the False Discovery proportion
Journal of the American Statistical Association, 2006Co-Authors: Christopher R. Genovese, Larry WassermanAbstract:Multiple testing methods to control the False Discovery rate, the expected proportion of Falsely rejected null hypotheses among all rejections, have received much attention. It can be valuable to control not the mean of this False Discovery proportion (FDP), but rather the probability that the FDP exceeds a specified bound. In this article we construct a general class of methods for exceedance control of FDP based on inverting tests of uniformity. The method also produces a confidence envelope for the FDP as a function of rejection threshold. We discuss how to select a procedure with good power.
-
a stochastic process approach to False Discovery control
Annals of Statistics, 2004Co-Authors: Christopher R. Genovese, Larry WassermaAbstract:This paper extends the theory of False Discovery rates (FDR) pioneered by Benjamini and Hochberg [J. Roy. Statist. Soc. Ser B 57 (1995) 289-300]. We develop a framework in which the False Discovery Proportion (FDP)-the number of False rejections divided by the number of rejections-is treated as a stochastic process. After obtaining the limiting distribution of the process, we demonstrate the validity of a class of procedures for controlling the False Discovery Rate (the expected FDP). We construct a confidence envelope for the whole FDP process. From these envelopes we derive confidence thresholds, for controlling the quantiles of the distribution of the FDP as well as controlling the number of False discoveries. We also investigate methods for estimating the p-value distribution.
-
operating characteristics and extensions of the False Discovery rate procedure
Journal of The Royal Statistical Society Series B-statistical Methodology, 2002Co-Authors: Christopher R. Genovese, Larry WassermanAbstract:We investigate the operating characteristics of the Benjamini-Hochberg False Discovery rate procedure for multiple testing. This is a distribution-free method that controls the expected fraction of Falsely rejected null hypotheses among those rejected. The paper provides a framework for understanding more about this procedure. We first study the asymptotic properties of the `deciding point' "D" that determines the critical "p"-value. From this, we obtain explicit asymptotic expressions for a particular risk function. We introduce the dual notion of False non-rejections and we consider a risk function that combines the False Discovery rate and False non-rejections. We also consider the optimal procedure with respect to a measure of conditional risk. Copyright 2002 Royal Statistical Society.
-
operating characteristics and extensions of the False Discovery rate procedure
Journal of The Royal Statistical Society Series B-statistical Methodology, 2002Co-Authors: Christopher R. Genovese, Larry WassermanAbstract:Summary. We investigate the operating characteristics of the Benjamini–Hochberg False Discovery rate procedure for multiple testing. This is a distribution-free method that controls the expected fraction of Falsely rejected null hypotheses among those rejected. The paper provides a framework for understanding more about this procedure. We first study the asymptotic properties of the ‘deciding point’ D that determines the critical p-value. From this, we obtain explicit asymptotic expressions for a particular risk function. We introduce the dual notion of False non-rejections and we consider a risk function that combines the False Discovery rate and False non-rejections. We also consider the optimal procedure with respect to a measure of conditional risk.
