The Experts below are selected from a list of 98370 Experts worldwide ranked by ideXlab platform
Yoav Benjamini - One of the best experts on this subject based on the ideXlab platform.
-
weighted False Discovery Rate controlling procedures for clinical trials
Biostatistics, 2017Co-Authors: Yoav Benjamini, Rami CohenAbstract:Having identified that the lack of replicability of results in earlier phases of clinical medical research stems largely from unattended selective inference, we offer a new hierarchical weighted False Discovery Rate controlling testing procedure alongside the single-level weighted procedure. These address the special structure of clinical research, where the comparisons of treatments involve both primary and secondary endpoints, by assigning weights that reflect the relative importance of the endpoints in the error being controlled. In the hierarchical method, the primary endpoints and a properly weighted intersection hypothesis that represents all secondary endpoints are tested. Should the intersection hypothesis be among the rejected, individual secondary endpoints are tested. We identify configurations where each of the two procedures has the advantage. Both offer higher power than competing hierarchical (gatekeeper) familywise error-Rate controlling procedures being used for drug approval. By their design, the advantage of the proposed methods is the increased power to discover effects on secondary endpoints, without giving up the rigor of addressing their multiplicity.
-
Wiley StatsRef: Statistics Reference Online - False Discovery Rate
Wiley StatsRef: Statistics Reference Online, 2014Co-Authors: Felix Abramovich, Yoav BenjaminiAbstract:For the multiple hypotheses testing problem consider the proportion of Falsely rejected hypotheses (False discoveries) among the total number of rejections. The expected value of this proportion, called the False Discovery Rate (FDR), is a useful criterion to control as an alternative to the traditional familywise error Rate (FWE) that suffers from low power properties when the number of tested hypotheses is large. In a way, controlling FDR is adaptively inbetween ignoring multiplicity altogether and a conservative control of FWE. Several FDR controlling procedures are presented and others are reviewed. Various extensions and applications of the FDR are discussed. Keywords: bonferroni-type procedures; familywise error Rate; False Discovery Rate; multiple hypotheses testing; p-values
-
discovering the False Discovery Rate
Journal of The Royal Statistical Society Series B-statistical Methodology, 2010Co-Authors: Yoav BenjaminiAbstract:I describe the background for the paper 'Controlling the False Discovery Rate: a new and powerful approach to multiple comparisons' by Benjamini and Hochberg that was published in the "Journal of the Royal Statistical Society", Series B, in 1995. I review the progress since made on the False Discovery Rate, as well as the major conceptual developments that followed. Copyright (c) 2010 Royal Statistical Society.
-
adaptive linear step up procedures that control the False Discovery Rate
Biometrika, 2006Co-Authors: Yoav Benjamini, Abba M Krieger, Daniel YekutieliAbstract:The linear step-up multiple testing procedure controls the False Discovery Rate at the desired level q for independent and positively dependent test statistics. When all null hypotheses are true, and the test statistics are independent and continuous, the bound is sharp. When some of the null hypotheses are not true, the procedure is conservative by a factor which is the proportion m-sub-0/m of the true null hypotheses among the hypotheses. We provide a new two-stage procedure in which the linear step-up procedure is used in stage one to estimate m-sub-0, providing a new level q′ which is used in the linear step-up procedure in the second stage. We prove that a general form of the two-stage procedure controls the False Discovery Rate at the desired level q. This framework enables us to study analytically the properties of other procedures that exist in the literature. A simulation study is presented that shows that two-stage adaptive procedures improve in power over the original procedure, mainly because they provide tighter control of the False Discovery Rate. We further study the performance of the current suggestions, some variations of the procedures, and previous suggestions, in the case where the test statistics are positively dependent, a case for which the original procedure controls the False Discovery Rate. In the setting studied here the newly proposed two-stage procedure is the only one that controls the False Discovery Rate. The procedures are illustRated with two examples of biological importance. Copyright 2006, Oxford University Press.
-
adapting to unknown sparsity by controlling the False Discovery Rate
Annals of Statistics, 2006Co-Authors: Felix Abramovich, Yoav Benjamini, David L Donoho, Iain M JohnstoneAbstract:We attempt to recover an n-dimensional vector observed in white noise, where n is large and the vector is known to be sparse, but the degree of sparsity is unknown. We consider three different ways of defining sparsity of a vector: using the fraction of nonzero terms; imposing power-law decay bounds on the ordered entries; and controlling the lp norm for p small. We obtain a procedure which is asymptotically minimax for l r loss, simultaneously throughout a range of such sparsity classes. The optimal procedure is a data-adaptive thresholding scheme, driven by control of the False Discovery Rate (FDR). FDR control is a relatively recent innovation in simultaneous testing, ensuring that at most a certain fraction of the rejected null hypotheses will correspond to False rejections. In our treatment, the FDR control parameter qn also plays a determining role in asymptotic minimaxity. If q = lim qn ∈ [0,1/2] and also qn > γ/log(n) we get sharp asymptotic minimaxity, simultaneously, over a wide range of sparse parameter spaces and loss functions. On the other hand, q = lim qn ∈ (1/2,1], forces the risk to exceed the minimax risk by a factor growing with q. To our knowledge, this relation between ideas in simultaneous inference and asymptotic decision theory is new. Our work provides a new perspective on a class of model selection rules which has been introduced recently by several authors. These new rules impose complexity penalization of the form 2 � log( potential model size / actual model size ). We exhibit a close connection with FDR-controlling procedures under stringent control of the False Discovery Rate.
Daniel Yekutieli - One of the best experts on this subject based on the ideXlab platform.
-
hierarchical False Discovery Rate controlling methodology
Journal of the American Statistical Association, 2008Co-Authors: Daniel YekutieliAbstract:We discuss methodology for controlling the False Discovery Rate (FDR) in complex large-scale studies that involve testing multiple families of hypotheses; the tested hypotheses are arranged in a tree of disjoint subfamilies, and the subfamilies of hypotheses are hierarchically tested by the Benjamini and Hochberg FDR-controlling (BH) procedure. We derive an approximation for the multiple family FDR for independently distributed test statistics: q, the level at which the BH procedure is applied, times the number of families tested plus the number of discoveries, divided by the number of discoveries plus 1. We provide a universal bound for the FDR of the discoveries in the new hierarchical testing approach, 2 × 1.44 × q, and demonstRate in simulations that when the data has an hierarchical structure the new testing approach can be considerably more powerful than the BH procedure.
-
Hierarchical False Discovery Rate–Controlling Methodology
Journal of the American Statistical Association, 2008Co-Authors: Daniel YekutieliAbstract:We discuss methodology for controlling the False Discovery Rate (FDR) in complex large-scale studies that involve testing multiple families of hypotheses; the tested hypotheses are arranged in a tree of disjoint subfamilies, and the subfamilies of hypotheses are hierarchically tested by the Benjamini and Hochberg FDR-controlling (BH) procedure. We derive an approximation for the multiple family FDR for independently distributed test statistics: q, the level at which the BH procedure is applied, times the number of families tested plus the number of discoveries, divided by the number of discoveries plus 1. We provide a universal bound for the FDR of the discoveries in the new hierarchical testing approach, 2 × 1.44 × q, and demonstRate in simulations that when the data has an hierarchical structure the new testing approach can be considerably more powerful than the BH procedure.
-
adaptive linear step up procedures that control the False Discovery Rate
Biometrika, 2006Co-Authors: Yoav Benjamini, Abba M Krieger, Daniel YekutieliAbstract:The linear step-up multiple testing procedure controls the False Discovery Rate at the desired level q for independent and positively dependent test statistics. When all null hypotheses are true, and the test statistics are independent and continuous, the bound is sharp. When some of the null hypotheses are not true, the procedure is conservative by a factor which is the proportion m-sub-0/m of the true null hypotheses among the hypotheses. We provide a new two-stage procedure in which the linear step-up procedure is used in stage one to estimate m-sub-0, providing a new level q′ which is used in the linear step-up procedure in the second stage. We prove that a general form of the two-stage procedure controls the False Discovery Rate at the desired level q. This framework enables us to study analytically the properties of other procedures that exist in the literature. A simulation study is presented that shows that two-stage adaptive procedures improve in power over the original procedure, mainly because they provide tighter control of the False Discovery Rate. We further study the performance of the current suggestions, some variations of the procedures, and previous suggestions, in the case where the test statistics are positively dependent, a case for which the original procedure controls the False Discovery Rate. In the setting studied here the newly proposed two-stage procedure is the only one that controls the False Discovery Rate. The procedures are illustRated with two examples of biological importance. Copyright 2006, Oxford University Press.
-
quantitative trait loci analysis using the False Discovery Rate
Genetics, 2005Co-Authors: Yoav Benjamini, Daniel YekutieliAbstract:False Discovery Rate control has become an essential tool in any study that has a very large multiplicity problem. False Discovery Rate-controlling procedures have also been found to be very effective in QTL analysis, ensuring reproducible results with few Falsely discovered linkages and offering increased power to discover QTL, although their acceptance has been slower than in microarray analysis, for example. The reason is partly because the methodological aspects of applying the False Discovery Rate to QTL mapping are not well developed. Our aim in this work is to lay a solid foundation for the use of the False Discovery Rate in QTL mapping. We review the False Discovery Rate criterion, the appropriate interpretation of the FDR, and alternative formulations of the FDR that appeared in the statistical and genetics literature. We discuss important features of the FDR approach, some stemming from new developments in FDR theory and methodology, which deem it especially useful in linkage analysis. We review False Discovery Rate-controlling procedures—the BH, the resampling procedure, and the adaptive two-stage procedure—and discuss the validity of these procedures in single- and multiple-trait QTL mapping. Finally we argue that the control of the False Discovery Rate has an important role in suggesting, indicating the significance of, and confirming QTL and present guidelines for its use.
-
identifying differentially expressed genes using False Discovery Rate controlling procedures
Bioinformatics, 2003Co-Authors: Anat Reiner, Daniel Yekutieli, Yoav BenjaminiAbstract:Motivation: DNA microarrays have recently been used for the purpose of monitoring expression levels of thousands of genes simultaneously and identifying those genes that are differentially expressed. The probability that a False identification (type I error) is committed can increase sharply when the number of tested genes gets large. Correlation between the test statistics attributed to gene co-regulation and dependency in the measurement errors of the gene expression levels further complicates the problem. In this paper we address this very large multiplicity problem by adopting the False Discovery Rate (FDR) controlling approach. In order to address the dependency problem, we present three resampling-based FDR controlling procedures, that account for the test statistics distribution, and compare their performance to that of the
James G Scott - One of the best experts on this subject based on the ideXlab platform.
-
False Discovery Rate smoothing
Journal of the American Statistical Association, 2018Co-Authors: Wesley Tansey, Oluwasanmi Koyejo, Russell A Poldrack, James G ScottAbstract:ABSTRACTWe present False Discovery Rate (FDR) smoothing, an empirical-Bayes method for exploiting spatial structure in large multiple-testing problems. FDR smoothing automatically finds spatially localized regions of significant test statistics. It then relaxes the threshold of statistical significance within these regions, and tightens it elsewhere, in a manner that controls the overall False Discovery Rate at a given level. This results in increased power and cleaner spatial separation of signals from noise. The approach requires solving a nonstandard high-dimensional optimization problem, for which an efficient augmented-Lagrangian algorithm is presented. In simulation studies, FDR smoothing exhibits state-of-the-art performance at modest computational cost. In particular, it is shown to be far more robust than existing methods for spatially dependent multiple testing. We also apply the method to a dataset from an fMRI experiment on spatial working memory, where it detects patterns that are much more b...
-
False Discovery Rate regression an application to neural synchrony detection in primary visual cortex
Journal of the American Statistical Association, 2015Co-Authors: James G Scott, Ryan C Kelly, Matthew A Smith, Pengcheng Zhou, Robert E KassAbstract:This article introduces False Discovery Rate regression, a method for incorporating covariate information into large-scale multiple-testing problems. FDR regression estimates a relationship between test-level covariates and the prior probability that a given observation is a signal. It then uses this estimated relationship to inform the outcome of each test in a way that controls the overall False Discovery Rate at a prespecified level. This poses many subtle issues at the interface between inference and computation, and we investigate several variations of the overall approach. Simulation evidence suggests that: (1) when covariate effects are present, FDR regression improves power for a fixed False-Discovery Rate; and (2) when covariate effects are absent, the method is robust, in the sense that it does not lead to inflated error Rates. We apply the method to neural recordings from primary visual cortex. The goal is to detect pairs of neurons that exhibit fine-time-scale interactions, in the sense that they fire together more often than expected due to chance. Our method detects roughly 50% more synchronous pairs versus a standard FDR-controlling analysis. The companion R package FDRreg implements all methods described in the article. Supplementary materials for this article are available online.
-
False Discovery Rate smoothing
arXiv: Methodology, 2014Co-Authors: Wesley Tansey, Oluwasanmi Koyejo, Russell A Poldrack, James G ScottAbstract:We present False Discovery Rate smoothing, an empirical-Bayes method for exploiting spatial structure in large multiple-testing problems. FDR smoothing automatically finds spatially localized regions of significant test statistics. It then relaxes the threshold of statistical significance within these regions, and tightens it elsewhere, in a manner that controls the overall False-Discovery Rate at a given level. This results in increased power and cleaner spatial separation of signals from noise. The approach requires solving a non-standard high-dimensional optimization problem, for which an efficient augmented-Lagrangian algorithm is presented. In simulation studies, FDR smoothing exhibits state-of-the-art performance at modest computational cost. In particular, it is shown to be far more robust than existing methods for spatially dependent multiple testing. We also apply the method to a data set from an fMRI experiment on spatial working memory, where it detects patterns that are much more biologically plausible than those detected by standard FDR-controlling methods. All code for FDR smoothing is publicly available in Python and R.
-
False Discovery Rate regression an application to neural synchrony detection in primary visual cortex
arXiv: Methodology, 2013Co-Authors: James G Scott, Ryan C Kelly, Matthew A Smith, Pengcheng Zhou, Robert E KassAbstract:Many approaches for multiple testing begin with the assumption that all tests in a given study should be combined into a global False-Discovery-Rate analysis. But this may be inappropriate for many of today's large-scale screening problems, where auxiliary information about each test is often available, and where a combined analysis can lead to poorly calibRated error Rates within different subsets of the experiment. To address this issue, we introduce an approach called False-Discovery-Rate regression that directly uses this auxiliary information to inform the outcome of each test. The method can be motivated by a two-groups model in which covariates are allowed to influence the local False Discovery Rate, or equivalently, the posterior probability that a given observation is a signal. This poses many subtle issues at the interface between inference and computation, and we investigate several variations of the overall approach. Simulation evidence suggests that: (1) when covariate effects are present, FDR regression improves power for a fixed False-Discovery Rate; and (2) when covariate effects are absent, the method is robust, in the sense that it does not lead to inflated error Rates. We apply the method to neural recordings from primary visual cortex. The goal is to detect pairs of neurons that exhibit fine-time-scale interactions, in the sense that they fire together more often than expected due to chance. Our method detects roughly 50% more synchronous pairs versus a standard FDR-controlling analysis. The companion R package FDRreg implements all methods described in the paper.
Robert E Kass - One of the best experts on this subject based on the ideXlab platform.
-
False Discovery Rate regression an application to neural synchrony detection in primary visual cortex
Journal of the American Statistical Association, 2015Co-Authors: James G Scott, Ryan C Kelly, Matthew A Smith, Pengcheng Zhou, Robert E KassAbstract:This article introduces False Discovery Rate regression, a method for incorporating covariate information into large-scale multiple-testing problems. FDR regression estimates a relationship between test-level covariates and the prior probability that a given observation is a signal. It then uses this estimated relationship to inform the outcome of each test in a way that controls the overall False Discovery Rate at a prespecified level. This poses many subtle issues at the interface between inference and computation, and we investigate several variations of the overall approach. Simulation evidence suggests that: (1) when covariate effects are present, FDR regression improves power for a fixed False-Discovery Rate; and (2) when covariate effects are absent, the method is robust, in the sense that it does not lead to inflated error Rates. We apply the method to neural recordings from primary visual cortex. The goal is to detect pairs of neurons that exhibit fine-time-scale interactions, in the sense that they fire together more often than expected due to chance. Our method detects roughly 50% more synchronous pairs versus a standard FDR-controlling analysis. The companion R package FDRreg implements all methods described in the article. Supplementary materials for this article are available online.
-
False Discovery Rate regression an application to neural synchrony detection in primary visual cortex
arXiv: Methodology, 2013Co-Authors: James G Scott, Ryan C Kelly, Matthew A Smith, Pengcheng Zhou, Robert E KassAbstract:Many approaches for multiple testing begin with the assumption that all tests in a given study should be combined into a global False-Discovery-Rate analysis. But this may be inappropriate for many of today's large-scale screening problems, where auxiliary information about each test is often available, and where a combined analysis can lead to poorly calibRated error Rates within different subsets of the experiment. To address this issue, we introduce an approach called False-Discovery-Rate regression that directly uses this auxiliary information to inform the outcome of each test. The method can be motivated by a two-groups model in which covariates are allowed to influence the local False Discovery Rate, or equivalently, the posterior probability that a given observation is a signal. This poses many subtle issues at the interface between inference and computation, and we investigate several variations of the overall approach. Simulation evidence suggests that: (1) when covariate effects are present, FDR regression improves power for a fixed False-Discovery Rate; and (2) when covariate effects are absent, the method is robust, in the sense that it does not lead to inflated error Rates. We apply the method to neural recordings from primary visual cortex. The goal is to detect pairs of neurons that exhibit fine-time-scale interactions, in the sense that they fire together more often than expected due to chance. Our method detects roughly 50% more synchronous pairs versus a standard FDR-controlling analysis. The companion R package FDRreg implements all methods described in the paper.
Hongzhe Li - One of the best experts on this subject based on the ideXlab platform.
-
Joint testing and False Discovery Rate control in high-dimensional multivariate regression
Biometrika, 2018Co-Authors: Hongzhe LiAbstract:SUMMARYMultivariate regression with high-dimensional covariates has many applications in genomic and genetic research, in which some covariates are expected to be associated with multiple responses. This paper considers joint testing for regression coefficients over multiple responses and develops simultaneous testing methods with False Discovery Rate control. The test statistic is based on inverse regression and bias-corrected group lasso estimates of the regression coefficients and is shown to have an asymptotic chi-squared null distribution. A row-wise multiple testing procedure is developed to identify the covariates associated with the responses. The procedure is shown to control the False Discovery proportion and False Discovery Rate at a prespecified level asymptotically. Simulations demonstRate the gain in power, relative to entrywise testing, in detecting the covariates associated with the responses. The test is applied to an ovarian cancer dataset to identify the microRNA regulators that regulate protein expression.
-
optimal False Discovery Rate control for dependent data
Statistics and Its Interface, 2011Co-Authors: John M Maris, Hongzhe LiAbstract:This paper considers the problem of optimal False Discovery Rate control when the test statistics are dependent. An optimal joint oracle procedure, which minimizes the False non-Discovery Rate subject to a constraint on the False Discovery Rate is developed. A data-driven marginal plug-in procedure is then proposed to approximate the optimal joint procedure for multivariate normal data. It is shown that the marginal procedure is asymptotically optimal for multivariate normal data with a short-range dependent covariance structure. Numerical results show that the marginal procedure controls False Discovery Rate and leads to a smaller False non-Discovery Rate than several commonly used p-value based False Discovery Rate controlling methods. The procedure is illustRated by an application to a genome-wide association study of neuroblastoma and it identifies a few more genetic variants that are potentially associated with neuroblastoma than several p-value-based False Discovery Rate controlling procedures.